-- Logs begin at Thu 2022-09-01 13:07:06 UTC, end at Thu 2022-09-01 14:26:15 UTC. -- Sep 01 13:42:10.581310 dv-cinder-852211-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 13:42:10.588909 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 13:42:10 2022] *** Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: nodename: dv-cinder-852211-7-iscsi Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: machine: x86_64 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: clock source: unix Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: pcre jit disabled Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: detected number of CPU cores: 4 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: current working directory: / Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: detected binary path: /usr/bin/uwsgi-core Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: your processes number limit is 47680 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: your memory page size is 4096 bytes Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: detected max file descriptor number: 2048 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: lock engine: pthread robust mutexes Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: thunder lock: enabled Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 13:42:10.593864 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 13:42:10.635954 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: Python main interpreter initialized at 0x56447b6d89c0 Sep 01 13:42:10.635954 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: python threads support enabled Sep 01 13:42:10.635954 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: your server socket listen backlog is limited to 100 connections Sep 01 13:42:10.635954 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: your mercy for graceful operations on workers is 90 seconds Sep 01 13:42:10.636356 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 13:42:10.636430 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: *** Operational MODE: preforking *** Sep 01 13:42:10.636711 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: *** uWSGI is running in multiple interpreter mode *** Sep 01 13:42:10.636711 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: spawned uWSGI master process (pid: 81692) Sep 01 13:42:10.636917 dv-cinder-852211-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 13:42:10.641929 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: spawned uWSGI worker 1 (pid: 81705, cores: 1) Sep 01 13:42:10.641929 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: spawned uWSGI worker 2 (pid: 81706, cores: 1) Sep 01 13:42:10.641929 dv-cinder-852211-7-iscsi devstack@keystone.service[81692]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 13:42:15.088487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:42:15.088487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: from cryptography.utils import int_from_bytes Sep 01 13:42:15.089314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:42:15.089314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: from cryptography.utils import int_from_bytes Sep 01 13:42:15.091374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:42:15.091374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: from cryptography.utils import int_from_bytes Sep 01 13:42:15.091888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:42:15.091888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: from cryptography.utils import int_from_bytes Sep 01 13:42:15.148113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 13:42:15.148364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 13:42:15.148467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 13:42:15.148555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 13:42:15.148624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] command line args: [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 13:42:15.148683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] command line args: [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 13:42:15.148751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 13:42:15.148833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 13:42:15.148908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ================================================================================ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 13:42:15.148973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ================================================================================ {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 13:42:15.149175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] admin_token = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.149338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] admin_token = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.149457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config_dir = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.149544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] config_dir = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.149657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.149830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config_source = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] config_source = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] debug = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] debug = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.150769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_config_append = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.151999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_config_append = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_dir = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_dir = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_file = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.152889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.153947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.154972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.155925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.156129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.156226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] publish_errors = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.156544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.157069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.157184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] publish_errors = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.157291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.157516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.157635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.157831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] standard_threads = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.158885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] standard_threads = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] use_journal = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] use_json = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.159999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] use_stderr = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.160113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_journal = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.160231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] use_syslog = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.160370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_json = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.160370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_stderr = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.161104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.161218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_syslog = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.161340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.161458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:42:15.161583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.161700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.161933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.162996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.163965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.164880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.165822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.174500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.174649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.174770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.174878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.175729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.176046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.176443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.176580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.176762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.176884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.176993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.177888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.178993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179066 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.179970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.180996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.181962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182319 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182450 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.182928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.183887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.184947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.185978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.186047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.186112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.186784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.186912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.187976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.188960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.189985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.190992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191066 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.external = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.password = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] auth.token = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.191941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.external = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.password = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.token = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.192955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.193982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.194938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.195940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.196934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.197977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.198945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.199923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.200978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.201880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.202978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.203936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.204989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.205861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.206945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.207920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.208999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.209974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.210934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.211936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.212905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.213868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.214975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.215948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216671 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.216918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.217843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] role.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] role.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.218829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219319 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.219967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.220934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.221771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.222990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.223980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.224898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.225865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.226887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.227847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.228965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.229988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.230991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.connection = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.231977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.232923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.233523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.233686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.233819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.234846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.235952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.236031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.236185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.236273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.236357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81705) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 13:42:15.236520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.236761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.237003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.237253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.237496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.237746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.238012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.238268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.238541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.238823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.239102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.239405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.239677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:42:15.239855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 13:42:15.267641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.267871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.268103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.269816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.270071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.270299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.276801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.279184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.315183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.315463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.315927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.316166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.383044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.383773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.432779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.433035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.434595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.434838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.594744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.594975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.595207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.595430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.595647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.595868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.601842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.602111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.602338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.602560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.602778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.602995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.620015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.620291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.620470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.620690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.620910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.621066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.621155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.621306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.621400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.621537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.621766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.622016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.622239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.622459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.632950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.635185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.647795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.647971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.648953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.649078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81705) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.649224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:42:15.678593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.678779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:42:15.679128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.679326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:42:15.679434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.679554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:42:15.679639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.679848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.680106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.680194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.680363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.680447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:42:15.680646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.680917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.681185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.681451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.690231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.690511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.690781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.690938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.691065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.691208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.691487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.691775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.698057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.698770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.700937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.701224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.701510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.701678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.701811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.702954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.703889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.704053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.704334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.704610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.727016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.727887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.728482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.728764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.729370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.729658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.733524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.733952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.734305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.734712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.735011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.735297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.740779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.741065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.741766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.742080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.745039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.745346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.745646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.745968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.746243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.746355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.746540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.746838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.747134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.747424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.747707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.747983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.748258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.748421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.749252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.749668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.766649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.766935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.767338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.767639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.770871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.771495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.773994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.774299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.774723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.775028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.778660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.779201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.781806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.782387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.784895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.790971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.791041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.791041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.791041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.802237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.802535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.802802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.802943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.803065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.803190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.803404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.803672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.809802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.810623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.812073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.812713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.813627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.813936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.819165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.819165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.819682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.819682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.819682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.820057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.820183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.820283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.820362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.820438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.820438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.820438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.829696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.830026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.830311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.830398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.830592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.830681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.830876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.831063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.831154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.831353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.831439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.831644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.831941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.832240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.847039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.847920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.850102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.850892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.852990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.853785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.854550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.855275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.856019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.856301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.856590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.861977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.861977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.861977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.863304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.864238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.864803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.865740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.867689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.867976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.868261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.868545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.868755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.869059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.869370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.869636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.877806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.878124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.878337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.878445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.878644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.878931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.885403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81705) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.885632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.885916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:42:15.886052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.886215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.886322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.886504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.886586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.886764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.886861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.887948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.888930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81705) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.889013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.889216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.890051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:42:15.924552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56447b6d89c0 pid: 81705 (default app) Sep 01 13:42:15.925030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56447b6d89c0 pid: 81706 (default app) Sep 01 13:42:15.928359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f115574-be5d-41db-b21e-8c89176fd4b0 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.928690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f115574-be5d-41db-b21e-8c89176fd4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.929013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f115574-be5d-41db-b21e-8c89176fd4b0 None None] PATH_INFO: `/v3` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.933202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Sep 1 13:42:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:23.573442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.573990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.574079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.581046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:23 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:23.586611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.587117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.587212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.660807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: INFO dbcounter [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] Registered counter for database keystone Sep 01 13:42:23.680547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG oslo_db.sqlalchemy.engines [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81705) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 13:42:23.687308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writer thread running {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 13:42:23.876720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG passlib.handlers.bcrypt [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81705) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 13:42:23.877020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG passlib.handlers.bcrypt [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81705) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 13:42:23.937421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:23.939086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:24.072169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 13:42:24.570342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.642926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: INFO dbcounter [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] Registered counter for database keystone Sep 01 13:42:24.665074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG oslo_db.sqlalchemy.engines [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81706) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 13:42:24.671698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writer thread running {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 13:42:24.795949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852c72-a187-460c-90d5-fa69a8a1e445 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.800410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.801039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.801325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.801602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.810233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.861682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING keystone.common.rbac_enforcer.enforcer [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 13:42:24.861970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5959c-5baa-47dc-b7a1-2b98a05192a7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.872340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 13:42:24.879355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.894206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0d85-8022-4a95-9104-e12bcf24133e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.898787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.899906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.900194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.900470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] PATH_INFO: `/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.902102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] RBAC: Authorizing `identity:get_project(project_id=1cbeb42d06b34679b1e34ad6fea416bc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.956707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING keystone.common.rbac_enforcer.enforcer [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 13:42:24.957059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72782f4b-3d0c-4e96-ae69-83fbbf519958 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.961778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:26.841139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295b673-598e-4d86-a4c2-ce44957191de None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.842266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295b673-598e-4d86-a4c2-ce44957191de None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.842266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295b673-598e-4d86-a4c2-ce44957191de None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.846506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:26.851996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.852430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.852518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.873840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:26.875664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:26.883677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 13:42:27.159165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295b673-598e-4d86-a4c2-ce44957191de None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.171976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295b673-598e-4d86-a4c2-ce44957191de None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.176208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df92e0d-ed25-4b29-a77a-b9463703e8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.176812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92e0d-ed25-4b29-a77a-b9463703e8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.177096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92e0d-ed25-4b29-a77a-b9463703e8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.177374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92e0d-ed25-4b29-a77a-b9463703e8fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.185089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df92e0d-ed25-4b29-a77a-b9463703e8fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.186954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df92e0d-ed25-4b29-a77a-b9463703e8fb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.196255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 1 13:42:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 13:42:27.202921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.212431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3f63-9dcb-4cc2-8358-923a9e69e783 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.217434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0cf1e-8e21-42de-bbd8-aa02bbce3de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.218321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cf1e-8e21-42de-bbd8-aa02bbce3de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.218603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cf1e-8e21-42de-bbd8-aa02bbce3de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.218879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cf1e-8e21-42de-bbd8-aa02bbce3de4 admin admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.220201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0cf1e-8e21-42de-bbd8-aa02bbce3de4 admin admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.222088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0cf1e-8e21-42de-bbd8-aa02bbce3de4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.223624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Sep 1 13:42:27 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:29.251498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5851470-e5b6-41de-9051-abf42313267b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.252079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5851470-e5b6-41de-9051-abf42313267b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.252079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5851470-e5b6-41de-9051-abf42313267b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.253346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:29.258771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.259491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.259491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.278312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:29.279575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:29.310899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919c19-c470-4b57-9867-7f42af570a59 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.311218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919c19-c470-4b57-9867-7f42af570a59 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.311504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919c19-c470-4b57-9867-7f42af570a59 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.312822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:29.318114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.318405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.318682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.361713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG passlib.handlers.bcrypt [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81706) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 13:42:29.362132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG passlib.handlers.bcrypt [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81706) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 13:42:29.364915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:29.412126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:29.413762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:29.519209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:29.640567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:29.665263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:29.747721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849c69e-4ecf-404d-8bca-7673afab552d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:29.783341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:29.783903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.784183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.784460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.789686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:29.796419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:29.805399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:29.811147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:29.812187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:29.813163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:29.894009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53b64e-7373-491d-b5d3-0bf330e3bea4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:29.896479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:29.898759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:29.917300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72038b7b-c6c2-4bc2-80bf-99402dd284b9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:29.930937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:29.931804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.932084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.932357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.953366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:29.954007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.954365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.954646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.955316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:29.955624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:29.957043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:29.969070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:29.974963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.006069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.007672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.079711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb8d39-f489-441e-9760-43470d374e3a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.114052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.114396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:42:30.114692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.114973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.115248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.116400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.117865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.119457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:29 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:30.119944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.123309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.125723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.140995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.222973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d034-b26e-4a8d-868d-c00e93beb84a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.244930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547666f-b2b7-4826-b292-b75f07448423 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.259380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.260253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.260538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.260813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.280214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.281703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.282186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.282849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.283116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.283398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.284552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.286044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.287621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:30.293477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:30.301639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:30.302685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:30.303485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:30.303485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:30.303485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:30.303485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:30.308813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.372201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.389713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.409508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56cd8b-c27d-4a05-a5f1-b7b7f6358d5f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.421917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3583ebf-cbc3-4a30-bc4e-93c834bc71a6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.444923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.445534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.445813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.446128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.446859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.domain.id=default, user.id=c6e62ba856084972b01892225cf77493) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.447165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.448603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93ccb-bc5a-4236-a623-de0c6fa82b92 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.456174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/role_assignments?user.id=c6e62ba856084972b01892225cf77493&scope.domain.id=default&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:30.457545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.458190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.458477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.458781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.459499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.459813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.461233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cfc35-c08a-46bb-9a53-42282a834c80 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.481409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:32.791372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce1783-cac6-448a-97f6-3a7c5bcc24dd None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.791915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce1783-cac6-448a-97f6-3a7c5bcc24dd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.791915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce1783-cac6-448a-97f6-3a7c5bcc24dd None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.793246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:32.812651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fea92-73ca-4278-ae4c-a4b0be68dd6d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.813141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fea92-73ca-4278-ae4c-a4b0be68dd6d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.813243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fea92-73ca-4278-ae4c-a4b0be68dd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.838909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6c3fea92-73ca-4278-ae4c-a4b0be68dd6d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:32.840128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6c3fea92-73ca-4278-ae4c-a4b0be68dd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:32.896077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2eba7-1d36-4f1c-ac10-a82bc499f620 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.896357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2eba7-1d36-4f1c-ac10-a82bc499f620 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.896605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2eba7-1d36-4f1c-ac10-a82bc499f620 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.898285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:32.903106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed564d63-d951-4009-92ef-bf28cf987642 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.903386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed564d63-d951-4009-92ef-bf28cf987642 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.904177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed564d63-d951-4009-92ef-bf28cf987642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.943178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ed564d63-d951-4009-92ef-bf28cf987642 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:32.944691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ed564d63-d951-4009-92ef-bf28cf987642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:32.978762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:33.058392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:33.177528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260b66-78fd-4898-bd8a-0ac44af49804 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.177812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260b66-78fd-4898-bd8a-0ac44af49804 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.178032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260b66-78fd-4898-bd8a-0ac44af49804 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.179796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 12/26] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:33.185518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.186099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.186570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.207381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:33.208877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:33.232057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645ca5f-49aa-42e4-ad9e-df50cb2cc41c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.232424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645ca5f-49aa-42e4-ad9e-df50cb2cc41c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.232936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645ca5f-49aa-42e4-ad9e-df50cb2cc41c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.234522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 13/27] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:33.239926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.240523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.240991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.273966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:33.275521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:33.286978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 15/28] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:33.360035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:33.391230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 14/29] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:33.454266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:33.458492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:33.602831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:33.607930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af358e8f-7935-4f2c-ab9e-a80217c81e77 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:33.647867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:33.648830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.649346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.650401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.651280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:33.652896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:33.749727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a221f-4c5c-44da-b79f-e8a784e19afa None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:33.788291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:33.788886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.789164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.789434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.802841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:33.810551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:33.811577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:33.812395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 15/30] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:33.813446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:33.856387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:33.917079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 16/31] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 13:42:33 2022] POST /identity/v3/roles => generated 219 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:33.917996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:33.930844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e938df-b9a3-4faf-9cd4-a48286f852a7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:33.972940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:33.973710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.974049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.974375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.987150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:33.988659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:34.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:34.009320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:34.011366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:34.011366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:34.011366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:34.011366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 16/32] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:34.011366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.087661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.119695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.136534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2f53-9aa1-444b-ad51-1c55f828e4ba None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.136721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b7f5f-6330-403e-98d2-823a44dadb68 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.181742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.182583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.183069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.183412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.184192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:34.184567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.186319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.186871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.187342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.187772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.190740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.192105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.198321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.202159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 17/33] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:34.203402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.279799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 17/34] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 13:42:33 2022] POST /identity/v3/roles => generated 225 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:34.281270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.319970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434c8b4-ef38-4fe5-ab81-3d1689ea74e2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.364767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.365845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.365845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.366125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.366849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:34.367154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.368645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.382966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 18/35] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:34.388997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.413154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.430149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78999a9-21dc-47f0-b09d-d0191c19c886 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.460143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.473997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad7af8-92b5-45dc-a687-6afc107158e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.475064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad7af8-92b5-45dc-a687-6afc107158e3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.475703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad7af8-92b5-45dc-a687-6afc107158e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.476247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad7af8-92b5-45dc-a687-6afc107158e3 None admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.477961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad7af8-92b5-45dc-a687-6afc107158e3 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.479872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad7af8-92b5-45dc-a687-6afc107158e3 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.482769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 18/36] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:34.484537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671da94-d1e7-4fd2-8029-2bb8add18d3d None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.485092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671da94-d1e7-4fd2-8029-2bb8add18d3d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.485809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671da94-d1e7-4fd2-8029-2bb8add18d3d None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.488044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 19/37] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:34.496519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671da94-d1e7-4fd2-8029-2bb8add18d3d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.538540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c83a41-7200-4643-9fa6-40fbcdda02aa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.582900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd988e-822c-464c-8e95-5cdfc1188c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.583528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd988e-822c-464c-8e95-5cdfc1188c2d None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.583812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd988e-822c-464c-8e95-5cdfc1188c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.584094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd988e-822c-464c-8e95-5cdfc1188c2d None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.585309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd988e-822c-464c-8e95-5cdfc1188c2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.586715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd988e-822c-464c-8e95-5cdfc1188c2d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.626924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671da94-d1e7-4fd2-8029-2bb8add18d3d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.661916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f454-02c6-4f37-9528-6d480cb21339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.662603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f454-02c6-4f37-9528-6d480cb21339 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.662972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f454-02c6-4f37-9528-6d480cb21339 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.663260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f454-02c6-4f37-9528-6d480cb21339 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.664481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242f454-02c6-4f37-9528-6d480cb21339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.665992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242f454-02c6-4f37-9528-6d480cb21339 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.667564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 20/38] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:34.668851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64948c-888f-483d-8e5c-d152b61cb35c None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.669671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64948c-888f-483d-8e5c-d152b61cb35c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.669972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64948c-888f-483d-8e5c-d152b61cb35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.688946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-0e64948c-888f-483d-8e5c-d152b61cb35c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:34.690518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0e64948c-888f-483d-8e5c-d152b61cb35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:34.751753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.763710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 21/39] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:34.765315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48363-0862-4102-bab7-317077b4ac1b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.765606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48363-0862-4102-bab7-317077b4ac1b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.765890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48363-0862-4102-bab7-317077b4ac1b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.767230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 22/40] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:34.767921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48363-0862-4102-bab7-317077b4ac1b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.840390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 19/41] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:34 2022] POST /identity/v3/projects => generated 301 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:34.842121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.842421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.842704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.864853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:34.866749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:34.904976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48363-0862-4102-bab7-317077b4ac1b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.944628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.957224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 20/42] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:34.957614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.959006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.959527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.960003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] PATH_INFO: `/v3/domains/default/users/c6e62ba856084972b01892225cf77493/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.962401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c6e62ba856084972b01892225cf77493, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.964183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.034570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.042019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 23/43] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Sep 1 13:42:34 2022] PUT /identity/v3/domains/default/users/c6e62ba856084972b01892225cf77493/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:35.169201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85803b-1a1c-4350-aae5-104add813ce8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.211413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.212041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.212327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.212605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.213308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.214888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.244334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.286070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 21/44] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 13:42:35 2022] POST /identity/v3/roles => generated 223 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:35.300705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.365486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f0fe-43ab-4554-8a6b-f0b6f3cda241 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.407125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f2795-ddee-4788-9ae0-dd1e7545c1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.407795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f2795-ddee-4788-9ae0-dd1e7545c1cb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.408080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f2795-ddee-4788-9ae0-dd1e7545c1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.408358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f2795-ddee-4788-9ae0-dd1e7545c1cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.409518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f2795-ddee-4788-9ae0-dd1e7545c1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.411029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f2795-ddee-4788-9ae0-dd1e7545c1cb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.412922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 24/45] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:35.418456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.486139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.650752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37039-3ae4-41e6-9f17-346c7666268c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.691772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676eb6f-0c2e-4ee6-b083-51f55b219822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.692396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676eb6f-0c2e-4ee6-b083-51f55b219822 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.692686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676eb6f-0c2e-4ee6-b083-51f55b219822 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.692969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676eb6f-0c2e-4ee6-b083-51f55b219822 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.693667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676eb6f-0c2e-4ee6-b083-51f55b219822 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.695170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676eb6f-0c2e-4ee6-b083-51f55b219822 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.881089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 22/46] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:35 2022] POST /identity/v3/projects => generated 312 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:37.011189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07654f2f-0ac9-47c2-8203-61ed7e6fe6f7 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.011727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07654f2f-0ac9-47c2-8203-61ed7e6fe6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.011811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07654f2f-0ac9-47c2-8203-61ed7e6fe6f7 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.013115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 25/47] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:37.019098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.019534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.019669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.038182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:37.039383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:37.109291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 23/48] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:37.381206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07654f2f-0ac9-47c2-8203-61ed7e6fe6f7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.601632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07654f2f-0ac9-47c2-8203-61ed7e6fe6f7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.620558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.639427 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.639995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.640277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.640557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.651011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:37.656398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:37.657339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:37.658108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:37.659560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 26/49] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:37.663844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.701932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.776600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88b62a-7f56-43a8-b412-24745ea40e96 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.813307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.813948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.814237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.814514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.815187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:37.815568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.816892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6147d2d-081a-47de-ad83-e0fac4d4fc50 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.830232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 24/50] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:37.831507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.843187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cc767-43b6-4613-b721-8d6812333c6a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.843938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cc767-43b6-4613-b721-8d6812333c6a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.844236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cc767-43b6-4613-b721-8d6812333c6a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.845585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 25/51] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:37.850992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.851215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.851495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.870641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:37.871881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:37.940339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3a80-42eb-4a8c-b761-eb3fc0ba3931 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.941454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 26/52] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:37.975189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.975831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.976119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.976401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.977538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.979038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.980630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 27/53] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:37.987360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.016078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.088674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899cce9-6c0e-46e3-b114-2023b7eede85 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.124904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.125548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.125835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.126136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.127255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.128703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.130303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 27/54] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:38.136103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.185551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.216198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.241633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb048e1-559a-41d8-b8e1-07cb553a7100 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.281432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.282097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.282381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.282659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.283429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.domain.id=default, user.id=c6e62ba856084972b01892225cf77493) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:38.283741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.285222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.291833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 28/55] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/role_assignments?user.id=c6e62ba856084972b01892225cf77493&scope.domain.id=default&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:38.319421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.327007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae162e9-5bb0-4055-b9c5-51ab4d502c8f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.368018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36e55c-9280-41e5-8ec3-5174a6fcae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.368684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e55c-9280-41e5-8ec3-5174a6fcae8c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.368974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e55c-9280-41e5-8ec3-5174a6fcae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.369287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e55c-9280-41e5-8ec3-5174a6fcae8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.370490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36e55c-9280-41e5-8ec3-5174a6fcae8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.372037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36e55c-9280-41e5-8ec3-5174a6fcae8c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.374034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 28/56] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:38.380172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.491436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f287ae-cee9-45a4-a5cc-29e0905db6d2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.503360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.527693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f66ab-d137-4048-97a2-62bc2d6a938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.528407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f66ab-d137-4048-97a2-62bc2d6a938d None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.528990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f66ab-d137-4048-97a2-62bc2d6a938d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.529355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f66ab-d137-4048-97a2-62bc2d6a938d None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.530108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f66ab-d137-4048-97a2-62bc2d6a938d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.531602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f66ab-d137-4048-97a2-62bc2d6a938d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.631818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 29/57] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 1 13:42:38 2022] POST /identity/v3/projects => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:40.566375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c8020-144d-4a51-adce-ac7458ef8696 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.566933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c8020-144d-4a51-adce-ac7458ef8696 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.566933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c8020-144d-4a51-adce-ac7458ef8696 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.568273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 29/58] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:40.573583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.574032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.574190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.592892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:40.594125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:40.663660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 30/59] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:40.940129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c8020-144d-4a51-adce-ac7458ef8696 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:41.040295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c8020-144d-4a51-adce-ac7458ef8696 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:41.074582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183231a-a9b2-4ac6-9b8c-7ff563de0822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:41.075816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183231a-a9b2-4ac6-9b8c-7ff563de0822 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.076198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183231a-a9b2-4ac6-9b8c-7ff563de0822 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.076482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183231a-a9b2-4ac6-9b8c-7ff563de0822 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.077609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183231a-a9b2-4ac6-9b8c-7ff563de0822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:41.079083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183231a-a9b2-4ac6-9b8c-7ff563de0822 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:41.080651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 30/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:41.086853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:41.178456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:41.187459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d694-db31-454e-acb6-98d8ade7043c None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:41.222003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcbc2d-493e-4525-8f81-79cad9dbcdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:41.222540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcbc2d-493e-4525-8f81-79cad9dbcdc9 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.222831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcbc2d-493e-4525-8f81-79cad9dbcdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.223112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcbc2d-493e-4525-8f81-79cad9dbcdc9 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.223781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcbc2d-493e-4525-8f81-79cad9dbcdc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:41.225281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcbc2d-493e-4525-8f81-79cad9dbcdc9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:41.330558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 31/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:42:41 2022] POST /identity/v3/users => generated 275 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:44.230780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09cfdb-14f9-41a1-8034-47e265ab4d2e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.279440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09cfdb-14f9-41a1-8034-47e265ab4d2e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.279440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09cfdb-14f9-41a1-8034-47e265ab4d2e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.279440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 31/62] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:44.279835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1904d-26a9-47ed-b59d-5960cdbf900f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.279835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1904d-26a9-47ed-b59d-5960cdbf900f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.279835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1904d-26a9-47ed-b59d-5960cdbf900f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.279835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-35f1904d-26a9-47ed-b59d-5960cdbf900f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:44.280849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-35f1904d-26a9-47ed-b59d-5960cdbf900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:44.382724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 32/63] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:44.726194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09cfdb-14f9-41a1-8034-47e265ab4d2e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.739549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99bcbb-7392-42bd-a236-3e66d622a7f5 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.740675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99bcbb-7392-42bd-a236-3e66d622a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.741158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99bcbb-7392-42bd-a236-3e66d622a7f5 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.743330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 33/64] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:44.750907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7681-4577-43ef-8ce7-050841eb7b92 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.751509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7681-4577-43ef-8ce7-050841eb7b92 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.752166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7681-4577-43ef-8ce7-050841eb7b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.779785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:44.787440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b98b7681-4577-43ef-8ce7-050841eb7b92 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:44.790097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b98b7681-4577-43ef-8ce7-050841eb7b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:44.882711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:44.884017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613a2d-6695-4e02-8493-d1626f67525b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.884475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09cfdb-14f9-41a1-8034-47e265ab4d2e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:44.884595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613a2d-6695-4e02-8493-d1626f67525b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.884761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613a2d-6695-4e02-8493-d1626f67525b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.890600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 35/66] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:44.892077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc84d5-b696-4b87-9974-4922e2f331a8 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.892366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc84d5-b696-4b87-9974-4922e2f331a8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.892648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc84d5-b696-4b87-9974-4922e2f331a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.912543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e3dc84d5-b696-4b87-9974-4922e2f331a8 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:44.914052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e3dc84d5-b696-4b87-9974-4922e2f331a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:44.917946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:44.931440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:44.932351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.932810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.933244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.945110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:44.950446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:44.951521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:44.952459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:44.954482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 32/67] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:44.960412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.995301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 36/68] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:45.061265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.071410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed9f1f-e5c9-4972-a7e0-3d1e7c686ce4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.085068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579473ad-90bc-47d2-a0e0-9f4a0a51d72f None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.085700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579473ad-90bc-47d2-a0e0-9f4a0a51d72f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.086207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579473ad-90bc-47d2-a0e0-9f4a0a51d72f None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.096057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 37/69] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:45.103721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.104373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.104869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.109674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.110331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.110620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.115654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.115654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:45.115654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.115654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.123371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 33/70] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:45.129850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.133640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:45.135139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:45.239767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230f94c-3b5e-4568-8845-54181e3a7839 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.265644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 38/71] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:45.275021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.278686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09498125-7f37-4c3e-a77e-dacdb7ff1531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.279572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498125-7f37-4c3e-a77e-dacdb7ff1531 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.280098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498125-7f37-4c3e-a77e-dacdb7ff1531 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.280532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498125-7f37-4c3e-a77e-dacdb7ff1531 None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.298947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09498125-7f37-4c3e-a77e-dacdb7ff1531 None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.300646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09498125-7f37-4c3e-a77e-dacdb7ff1531 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.302917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 34/72] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:45.309308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933da093-4563-411d-bd2d-4ca41d54fe22 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.309820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933da093-4563-411d-bd2d-4ca41d54fe22 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.310429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933da093-4563-411d-bd2d-4ca41d54fe22 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.311986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 35/73] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:45.312883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933da093-4563-411d-bd2d-4ca41d54fe22 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.327171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.358250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.397729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2f7e1-cf70-4f94-8980-f1a232ed6db7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.433823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e85920-007d-415e-a90c-1e478b46bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.434788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933da093-4563-411d-bd2d-4ca41d54fe22 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.435333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e85920-007d-415e-a90c-1e478b46bd04 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.435624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e85920-007d-415e-a90c-1e478b46bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.435902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e85920-007d-415e-a90c-1e478b46bd04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.448104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e85920-007d-415e-a90c-1e478b46bd04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-e1e85920-007d-415e-a90c-1e478b46bd04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:45.453204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:45.454214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:45.455010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 39/74] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:45.456555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.456555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.457084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.474685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.475477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.475771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.476088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.479295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:45.480508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:45.549819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.551348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.554695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 40/75] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:45.555457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.569162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.627627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.672698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc939a-6177-4b2e-aed5-535eb021b4af None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.676853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 36/76] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:45.677554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.713623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.714666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.715618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.715919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.728032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:45.734010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:45.734865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:45.735853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 41/77] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:45.736531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.808427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e668f2-5296-4383-a7b1-3bbedc3658f2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.820739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.848507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.849203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.849544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.849930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.850607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:45.850914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.852295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.855078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842de654-aa3c-4b6d-adbc-ae7e6af8f55a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.862809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 37/78] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:45.863717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.876507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.895639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.896318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.896608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.896888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.909839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:45.915008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:45.915874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:45.916806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 42/79] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:45.917771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.981923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091cb51-e09f-4794-96a4-54db7b98fd67 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.017854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.019019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.019019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.019310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.020556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.022047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.023021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caedafe-8033-4782-8fbe-b5046030c13b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.023712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 38/80] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.024500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.062350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.062976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.063265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.063547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.064224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:46.064542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.066005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.070033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.074722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 43/81] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.077026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.123351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.134585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d052-5a0b-44b7-b98a-691817ce108d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.173526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.174605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.175088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.175572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.177390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.180106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.183104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 39/82] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.184983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.192199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008049ed-01b4-450b-87e6-918047ba358c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.226978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.227619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.227896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.228168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.229397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.231443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.233343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 44/83] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.233887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.308339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54495987-8282-47a7-ab8e-451aae791af1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.318589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.338445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70a631-34f8-4c1b-902b-f74609f35f3b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.344622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.357687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.357687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.357687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.357687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:46.357687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.357687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.358228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.362904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 40/84] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.362904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.378488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.379327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.379472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.379744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.380367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=c2cca81b77904c7685856e765eb594be) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:46.380674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.382149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.388668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 45/85] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/role_assignments?user.id=c2cca81b77904c7685856e765eb594be&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.389578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.490563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c01da0-c8f4-414c-82da-91f67a09249c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.511335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3dde5-8357-49ae-8381-e0bd04e05e80 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.531668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.532343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.532641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.532920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.534998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.535585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.537222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 41/86] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.538042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.563933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.564826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.565554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.565915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.566899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.569076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.582779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.646465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb0aa-3a8a-4a32-9746-e3cebfc95f8d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.675792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 46/87] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:42:46 2022] POST /identity/v3/users => generated 289 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:46.676944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.709381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.710050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.710340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.710793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.725782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.814330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.815897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.835250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6935-5540-42b4-8a9c-4a7c019e1533 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.865263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.884171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.884886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.885180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.885707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.887708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.889254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.890922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 47/88] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.891718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.950556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 42/89] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:46.952162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.028339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.040168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8e2da-db65-48f9-a713-29f51ade37fb None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.092368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.092923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.093221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.093490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.111373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34efcb-c32e-4d15-b7f2-3639bae87b0e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.158989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.159208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.159501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.159788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.172160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.182157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.184397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.245148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.247612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.311563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.322396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 48/90] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:47.324675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.415425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 43/91] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:47.416969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.477283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.537948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea307ff-30e3-436d-b4cc-41d63b00ffa1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.610777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.611464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.611752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.612037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.613528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.615065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.616977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 49/92] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:47.617574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.619568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc329cf-eb05-444e-b97e-da6b2240c6a1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.690221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.690938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.691479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.691763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.695281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.696753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.700695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 44/93] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:47.701545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.762482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.842821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f9c5-f3f8-4a4d-86c5-6ceb7c8a78b8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.869229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25c3bb-e439-4f99-8446-8f2c9abd1064 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.878573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.912243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.913307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.913827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.914352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.915944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] RBAC: Adding query filter params (role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=c2cca81b77904c7685856e765eb594be) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:47.916473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.918867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.928485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 45/94] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/role_assignments?user.id=c2cca81b77904c7685856e765eb594be&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:47.929695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.931611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.932283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.932582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.932898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] PATH_INFO: `/v3/projects/efc34c1702f747d5829b764643d3201e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.935897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] RBAC: Authorizing `identity:get_project(project_id=efc34c1702f747d5829b764643d3201e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.937599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.939802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 50/95] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/projects/efc34c1702f747d5829b764643d3201e => generated 312 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:47.948154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:48.086234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eea46-0bd8-44a7-87cc-1af1e2a9e61d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:48.131625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:48.132646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.133140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.133694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.135038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=c6e62ba856084972b01892225cf77493) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:48.135730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:48.138060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be26b6-ab0c-4e71-b223-c8826cf562f0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:48.147017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 46/96] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/role_assignments?user.id=c6e62ba856084972b01892225cf77493&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:48.196251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90053d2b-7c06-49fd-90ee-34a8cc4b5373 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:48.235556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:48.241933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:48.243031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.243580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.244088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.245058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=efc34c1702f747d5829b764643d3201e, user.id=c2cca81b77904c7685856e765eb594be) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:48.245554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:48.247392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d51ec-43b8-4357-bebf-ac009fbb2c26 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:48.255007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 51/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/role_assignments?user.id=c2cca81b77904c7685856e765eb594be&scope.project.id=efc34c1702f747d5829b764643d3201e&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:48.702330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e6ed-51c8-4880-986d-4454ace8632b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.703421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e6ed-51c8-4880-986d-4454ace8632b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.703921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e6ed-51c8-4880-986d-4454ace8632b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.705996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 47/98] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:48.717730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.718774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.719309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.752195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:48.753784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:48.849480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 52/99] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:49.418632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e6ed-51c8-4880-986d-4454ace8632b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:49.458816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:49.548156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e6ed-51c8-4880-986d-4454ace8632b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:49.593845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ca9fa-9659-46c5-ba09-d61e86395d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:49.594524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca9fa-9659-46c5-ba09-d61e86395d9c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:49.594815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca9fa-9659-46c5-ba09-d61e86395d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:49.600264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca9fa-9659-46c5-ba09-d61e86395d9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:49.622232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ca9fa-9659-46c5-ba09-d61e86395d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-766ca9fa-9659-46c5-ba09-d61e86395d9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:49.627820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:49.628634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:49.629884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 48/100] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:49.635012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:49.790793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc929ca-2c4b-40c2-a99f-b2f4ab56eb49 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:49.829225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:49.843139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:49.843761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:49.844043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:49.844321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:49.849410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:49.849410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:49.849410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:49.856596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123283-597f-456a-a43c-df59707d0888 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:49.857161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123283-597f-456a-a43c-df59707d0888 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:49.857334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123283-597f-456a-a43c-df59707d0888 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:49.858581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 49/101] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:49.865795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee66e1-8563-476f-880d-c85136d4eb80 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:49.866096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 53/102] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:49.866251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee66e1-8563-476f-880d-c85136d4eb80 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:49.866413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee66e1-8563-476f-880d-c85136d4eb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:49.871763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:49.904846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1bee66e1-8563-476f-880d-c85136d4eb80 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:49.906132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1bee66e1-8563-476f-880d-c85136d4eb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:50.086269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 50/103] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 13:42:50.087521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e00c0-8969-4ba9-92dd-22d39be7d55d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.087809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e00c0-8969-4ba9-92dd-22d39be7d55d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.088088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e00c0-8969-4ba9-92dd-22d39be7d55d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.089571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 51/104] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:50.097571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.097876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.099791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.107170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a7dc4-4c2d-48dc-86d0-95cc61f7fb73 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.137871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:50.139325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:50.187964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.192297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.193246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.193661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.194127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.195593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:50.197249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:50.199189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 54/105] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:49 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:50.209308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.284136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 52/106] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:50.314323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.376286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa54b15-d3d8-4c27-8034-3c75eca286ae None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.424861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.426711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.427390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.427681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.427965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.429151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:50.432646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:50.432646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 55/107] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:50.443366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.647271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703c356-22ba-495d-90aa-88a434e47d65 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.648197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bdb35-0ea5-4d4b-8af2-34f805d2efb2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.702287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.709505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.709680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07220b-8bce-4b72-9299-eb113a96bf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.710212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07220b-8bce-4b72-9299-eb113a96bf0f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.710309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.710534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.710627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07220b-8bce-4b72-9299-eb113a96bf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.710766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/c2cca81b77904c7685856e765eb594be/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.718181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07220b-8bce-4b72-9299-eb113a96bf0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.718292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] RBAC: Authorizing `identity:create_grant(project_id=86ef03b588f047dc8dc8400f1ac175a7, user_id=c2cca81b77904c7685856e765eb594be, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:50.718292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:50.732833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07220b-8bce-4b72-9299-eb113a96bf0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9d07220b-8bce-4b72-9299-eb113a96bf0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:50.738563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:50.741397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:50.742522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 53/108] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:50.744443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2f598-517d-4165-b4b9-26eeddaf8d2f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.746095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2f598-517d-4165-b4b9-26eeddaf8d2f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.746634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2f598-517d-4165-b4b9-26eeddaf8d2f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.751717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 54/109] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:50.751717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2f598-517d-4165-b4b9-26eeddaf8d2f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.770799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.819365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 56/110] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:50 2022] PUT /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/c2cca81b77904c7685856e765eb594be/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:50.824018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.944182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2f598-517d-4165-b4b9-26eeddaf8d2f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.024324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.024942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.025227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.025505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.026713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:51.027032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.029402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8d858-88ef-4c0c-9d9c-8fa1c647eaa3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.041109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 55/111] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:51.042366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.042632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.042783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402fa2c-7fce-4dea-9a88-891dfa53da70 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.043520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.065454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:51.066702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:51.088302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28f1d1-5e35-4329-8774-d492e55e5562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.088953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28f1d1-5e35-4329-8774-d492e55e5562 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.089240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28f1d1-5e35-4329-8774-d492e55e5562 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.089521 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28f1d1-5e35-4329-8774-d492e55e5562 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.091290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28f1d1-5e35-4329-8774-d492e55e5562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.092756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28f1d1-5e35-4329-8774-d492e55e5562 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.094705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 57/112] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:51.100568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec92309-6060-45aa-9fa9-9d8ff826047f None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.100965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec92309-6060-45aa-9fa9-9d8ff826047f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.101261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec92309-6060-45aa-9fa9-9d8ff826047f None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.102970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 58/113] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:51.104185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec92309-6060-45aa-9fa9-9d8ff826047f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.167141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 56/114] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:51.168183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.171187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.181509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.226521 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec92309-6060-45aa-9fa9-9d8ff826047f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.263026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e0e7-6f5a-41d8-a567-f827fc119261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.263677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e0e7-6f5a-41d8-a567-f827fc119261 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.264046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e0e7-6f5a-41d8-a567-f827fc119261 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.264407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e0e7-6f5a-41d8-a567-f827fc119261 None admin] PATH_INFO: `/v3/users/5762bde7301740b685093f725e8969d8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.286201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e0e7-6f5a-41d8-a567-f827fc119261 None admin] RBAC: Authorizing `identity:get_user(user_id=5762bde7301740b685093f725e8969d8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.288678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e0e7-6f5a-41d8-a567-f827fc119261 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.291832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 59/115] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/users/5762bde7301740b685093f725e8969d8 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:51.293933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b0496-759d-4173-91a7-53afedd9b5ae None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.294367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b0496-759d-4173-91a7-53afedd9b5ae None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.294789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b0496-759d-4173-91a7-53afedd9b5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.305214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b01af-07c1-4a9d-bb3b-1be70da95afe None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.321230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-143b0496-759d-4173-91a7-53afedd9b5ae None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:51.322524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-143b0496-759d-4173-91a7-53afedd9b5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:51.349849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f21c6-ed31-4651-a21f-17ce4c847b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.351058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f21c6-ed31-4651-a21f-17ce4c847b6b None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.351602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f21c6-ed31-4651-a21f-17ce4c847b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.351853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f21c6-ed31-4651-a21f-17ce4c847b6b None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.352530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f21c6-ed31-4651-a21f-17ce4c847b6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.354839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f21c6-ed31-4651-a21f-17ce4c847b6b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.409235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 60/116] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:51.410609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ade240-fca4-4ebb-a50e-a9a448e9b999 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.410904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ade240-fca4-4ebb-a50e-a9a448e9b999 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.411209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ade240-fca4-4ebb-a50e-a9a448e9b999 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.412480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 61/117] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:51.413192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ade240-fca4-4ebb-a50e-a9a448e9b999 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.460524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 57/118] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:51 2022] POST /identity/v3/projects => generated 302 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:51.462141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.462676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.470101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.485617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.511011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:51.513401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:51.612332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ade240-fca4-4ebb-a50e-a9a448e9b999 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.657763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.670748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.674611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.674611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.674611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.677732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 58/119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:51.678937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.852381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f404c-48de-4cff-9664-a73dbf7c46fe None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.876269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.877786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.909421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.910490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.910980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.911455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.932251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:51.946500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:51.947810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:51.948967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:51.950386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:51.953269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 59/120] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:51.954433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.040220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 62/121] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:52.045754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.045937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.096451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede250d-40ad-4ea9-9065-3b1a06754011 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.148660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.149654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.150723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.151215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.164143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a79b7-df71-4107-a8e2-1bc9992ee122 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.166965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:52.173556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:52.174467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:52.175209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:52.177499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 60/122] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:52.178633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.204753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.205380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.205667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.206023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.214795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.218718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:52.223860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:52.228461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 63/123] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:52.229226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.297310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6ddb0-da99-429f-8a3f-614c62650839 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.307423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.334888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.335939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.336426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.336911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.337817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.338018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09c056-770b-4047-9e3c-107fbcebd549 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.338531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.340981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.351827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 61/124] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.359279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.379638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.380287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.380650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.380853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.382072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.383569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.389997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 64/125] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.389997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.459875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.485340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccfc47-8e11-4b32-8cb2-2f5acde89682 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.499920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36473989-eb91-4c7a-8128-70d92163ce51 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.526862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.528624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.529221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.529703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.530630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.531143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.533029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.540587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.541335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.541591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.541967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.542634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.543756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.545411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.545856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 62/126] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.547295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.555109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 65/127] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.556128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.585881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.667361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee0bea-c10b-4623-82bb-b45b497bc0ab None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.670302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f88b-3fe2-4b1f-a88e-fd216c38058b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.707465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.709353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.710378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.710918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.711447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.713019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.714100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.722300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.722449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.722568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.722568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.722568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] RBAC: Adding query filter params (role.id=1de11773e89446bc96f058fca76ee658, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=5762bde7301740b685093f725e8969d8) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.722568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.722808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 63/128] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.722808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.722923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.725604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 66/129] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/role_assignments?user.id=5762bde7301740b685093f725e8969d8&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=1de11773e89446bc96f058fca76ee658 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.729933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.854998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba629f1-0840-440c-bbe1-d93235b1da2d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.898786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.898786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.899640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.899640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.900712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.903062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.905275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 67/130] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.906324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.960613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.968305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ccc6b-c0d5-4260-aa76-200a10256ee1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.994297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:53.024416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.025352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.025679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.026241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.029672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.031246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.033842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 64/131] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:53.035142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.045795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c57ce1-f7c9-4f00-9fbb-f33a888324bc None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:53.091248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735c92a-dd38-4b55-b765-66c0ecf104bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.092026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c92a-dd38-4b55-b765-66c0ecf104bd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.092456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c92a-dd38-4b55-b765-66c0ecf104bd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.092944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c92a-dd38-4b55-b765-66c0ecf104bd None admin] PATH_INFO: `/v3/projects/efc34c1702f747d5829b764643d3201e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.094379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c92a-dd38-4b55-b765-66c0ecf104bd None admin] RBAC: Authorizing `identity:get_project(project_id=efc34c1702f747d5829b764643d3201e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.096039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c92a-dd38-4b55-b765-66c0ecf104bd None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.097820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 68/132] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/projects/efc34c1702f747d5829b764643d3201e => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:53.102204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46e72a-5dbf-44eb-b5da-96a253911ed4 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.102204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46e72a-5dbf-44eb-b5da-96a253911ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.102204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46e72a-5dbf-44eb-b5da-96a253911ed4 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.102204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 69/133] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:53.102539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46e72a-5dbf-44eb-b5da-96a253911ed4 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.199782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb8d7c-5024-4ac3-8cd8-5ea6ca01b589 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:53.248200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.249693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.250435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.250934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.252923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.255022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.257662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 65/134] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:53 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:53.258723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.298288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46e72a-5dbf-44eb-b5da-96a253911ed4 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:53.314635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:53.348950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:53.355858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d863c-4699-4116-b37b-b569c770c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.356807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d863c-4699-4116-b37b-b569c770c009 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.357262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d863c-4699-4116-b37b-b569c770c009 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.357621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d863c-4699-4116-b37b-b569c770c009 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.359293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d863c-4699-4116-b37b-b569c770c009 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.360628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d863c-4699-4116-b37b-b569c770c009 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.362767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 70/135] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:53 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:53.363963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.364347 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.364647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.388527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:53.391113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:53.413888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9768d5d-cc6b-44ab-a69b-9146b4edf96b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:53.457600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.458316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.458607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.458893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] PATH_INFO: `/v3/projects/efc34c1702f747d5829b764643d3201e/users/c2cca81b77904c7685856e765eb594be/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.460943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] RBAC: Authorizing `identity:create_grant(project_id=efc34c1702f747d5829b764643d3201e, user_id=c2cca81b77904c7685856e765eb594be, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.462669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.476390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 71/136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:53.482750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.554460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 66/137] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:53 2022] PUT /identity/v3/projects/efc34c1702f747d5829b764643d3201e/users/c2cca81b77904c7685856e765eb594be/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:53.555780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.656169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3acc7-8de2-4cf1-9fad-a0cfe82eb853 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:53.722684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:53.737186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.739041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.741035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.741467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/c2cca81b77904c7685856e765eb594be/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.747784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] RBAC: Authorizing `identity:create_grant(project_id=86ef03b588f047dc8dc8400f1ac175a7, user_id=c2cca81b77904c7685856e765eb594be, role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.751277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.762998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:53.774311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37409490-18a9-4199-ba8f-9f56988c25eb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:53.818968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ea4ca-6b73-44e5-8021-ea9d5020f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:53.819593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ea4ca-6b73-44e5-8021-ea9d5020f07a None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.819881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ea4ca-6b73-44e5-8021-ea9d5020f07a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.820165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ea4ca-6b73-44e5-8021-ea9d5020f07a None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/c6e62ba856084972b01892225cf77493/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.822280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ea4ca-6b73-44e5-8021-ea9d5020f07a None admin] RBAC: Authorizing `identity:create_grant(project_id=86ef03b588f047dc8dc8400f1ac175a7, user_id=c6e62ba856084972b01892225cf77493, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:53.823806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ea4ca-6b73-44e5-8021-ea9d5020f07a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:53.862826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 72/138] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:53 2022] PUT /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/c2cca81b77904c7685856e765eb594be/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:53.863637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.936493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 67/139] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:53 2022] PUT /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/c6e62ba856084972b01892225cf77493/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:54.009234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308034c-7963-40e3-a6c6-494cc3d244af None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.009983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308034c-7963-40e3-a6c6-494cc3d244af None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.009983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308034c-7963-40e3-a6c6-494cc3d244af None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.011392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 68/140] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:54.017243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.017608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.017854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.049700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f293f-4a66-4537-86c8-53d1bfc36c79 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.058931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:54.060212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:54.125234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.125815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.126115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.126387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.143697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:54.148776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:54.149959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:54.150916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 73/141] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:54.161731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.216952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 69/142] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:54.242275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.300413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414d2f-632a-4c25-bf43-7bacfb63def7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.341018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.341639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.341946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.342225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.342896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:54.343198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.344746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.354217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 74/143] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:54.363831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.515014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.533650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ae51-110c-4d93-b0a3-9eef0a19a7e9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.592940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.593887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.594368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.595026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.601437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.603101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.605744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 70/144] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:54.613699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.722175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.812885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1752b7-3edc-401a-adfc-dc98e6fee4a0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.834287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.894695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.895511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.895896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.896268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.960096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a5b4-dee4-487e-adf1-9f65845272ab None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.009333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.011086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.058776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.059635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.059933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.060227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.061516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.063062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.064673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 71/145] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:55.069999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.128900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.154958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 75/146] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:55.170021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.190154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.248465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d093b1-29e2-402b-b35f-d825359b3f0b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.304010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.304783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.305068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.305347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.306050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.307559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.376697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a85b5e-e83b-4631-b7b0-f503813c2a2a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.419888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e09f7a-c453-4972-a49f-cca43e67dccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.420089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 72/147] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:42:55 2022] POST /identity/v3/users => generated 283 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:55.420683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09f7a-c453-4972-a49f-cca43e67dccc None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.421015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09f7a-c453-4972-a49f-cca43e67dccc None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.421297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09f7a-c453-4972-a49f-cca43e67dccc None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.422805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09f7a-c453-4972-a49f-cca43e67dccc None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.424432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09f7a-c453-4972-a49f-cca43e67dccc None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.426231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 76/148] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:55.433057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.509773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47caa33-a79c-4ecd-b517-c1475cd88d86 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.510011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47caa33-a79c-4ecd-b517-c1475cd88d86 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.510289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47caa33-a79c-4ecd-b517-c1475cd88d86 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.511946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 77/149] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:55.518928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.519611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.520115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.544564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:55.546248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:55.598784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.652994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b5d8-0424-49b4-be19-d43fd002fc95 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.676331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 78/150] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:55.723842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.724473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.724759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.725036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.725792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=c2cca81b77904c7685856e765eb594be) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:55.726133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.732190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.735831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 73/151] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/role_assignments?user.id=c2cca81b77904c7685856e765eb594be&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.114082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.125428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.237406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8514dd4-7127-4ab1-a6a4-5369ba44d850 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.274887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.275538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.275825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.276096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.289108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:56.294286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:56.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:42:56.295882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:56.298900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 79/152] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:56.307035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.445302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94cef-87f3-4f10-b35d-6fbdb2c22322 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.492870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.493641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.494041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.494366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.495294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:56.496326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.498841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469f7dc-4795-4e52-aa13-248fd04c4376 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.509584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 74/153] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.516301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.645849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.669015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca864f-423b-46d3-97c3-f2914ae950d0 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.669325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca864f-423b-46d3-97c3-f2914ae950d0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.669592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca864f-423b-46d3-97c3-f2914ae950d0 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.671141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 75/154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:56.678185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a82ad-cd72-4fa7-850a-ff04ef2f4104 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.678495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a82ad-cd72-4fa7-850a-ff04ef2f4104 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.678724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a82ad-cd72-4fa7-850a-ff04ef2f4104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.688672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.696701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500a49-b326-42c2-87e9-093249cd67a4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.710509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-969a82ad-cd72-4fa7-850a-ff04ef2f4104 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:56.711733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-969a82ad-cd72-4fa7-850a-ff04ef2f4104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:56.753765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00f9da-d476-43b3-8317-f1b817f5cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.756037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00f9da-d476-43b3-8317-f1b817f5cf31 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.756037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00f9da-d476-43b3-8317-f1b817f5cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.756037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00f9da-d476-43b3-8317-f1b817f5cf31 None admin] PATH_INFO: `/v3/users/5762bde7301740b685093f725e8969d8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.757182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00f9da-d476-43b3-8317-f1b817f5cf31 None admin] RBAC: Authorizing `identity:get_user(user_id=5762bde7301740b685093f725e8969d8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.758854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00f9da-d476-43b3-8317-f1b817f5cf31 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.760508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 80/155] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/users/5762bde7301740b685093f725e8969d8 => generated 289 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.766010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bd7d1-2655-40e1-bd8f-efa3d4ff5e4f None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.766377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bd7d1-2655-40e1-bd8f-efa3d4ff5e4f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.766657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bd7d1-2655-40e1-bd8f-efa3d4ff5e4f None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.771394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 81/156] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:56.772312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bd7d1-2655-40e1-bd8f-efa3d4ff5e4f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.806138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 76/157] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:56.807506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774491-ff9e-4c33-b092-451cce52094e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.807796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774491-ff9e-4c33-b092-451cce52094e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.808100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774491-ff9e-4c33-b092-451cce52094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.826865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ad774491-ff9e-4c33-b092-451cce52094e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:56.828059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad774491-ff9e-4c33-b092-451cce52094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:56.910395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bd7d1-2655-40e1-bd8f-efa3d4ff5e4f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.912949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 77/158] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:56.914456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a76fc-e3bd-4b53-b5a2-18fa4fdab88a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.914950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a76fc-e3bd-4b53-b5a2-18fa4fdab88a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.915421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a76fc-e3bd-4b53-b5a2-18fa4fdab88a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.916885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 78/159] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:56.923364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.923946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.924431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.955048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:56.956092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.956795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.957063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:56.957164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.957337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.959348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.961630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.962663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 82/160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.971875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.979543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.060884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.081674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 79/161] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:57.145224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea2be0-55da-4d6b-ab86-661c4b660f4f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.192529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.193215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.193495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.193771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/5762bde7301740b685093f725e8969d8/roles/1de11773e89446bc96f058fca76ee658` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.195763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] RBAC: Authorizing `identity:create_grant(project_id=86ef03b588f047dc8dc8400f1ac175a7, user_id=5762bde7301740b685093f725e8969d8, role_id=1de11773e89446bc96f058fca76ee658)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:57.197231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:57.269555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 83/162] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:56 2022] PUT /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/users/5762bde7301740b685093f725e8969d8/roles/1de11773e89446bc96f058fca76ee658 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:42:57.304125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.307864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.406329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.474526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f9ac-1a60-4d6d-97d3-d04269dbb201 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.528346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.528900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.529188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.529468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.541295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:57.544522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e41e3-f2d2-491d-9081-dfc1bfe85e61 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:57.548033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:57.548988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 84/163] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:57.549776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.666609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b2ff-50ab-4425-888c-6e25c1d49865 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.676977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.677634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.677930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.678215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.693964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.711634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:57.719662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.720313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.720592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.720862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:57.721859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:57.722697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:57.723483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:42:57.725815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 80/164] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:57.726794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.750023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:57.753877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:57.754870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:42:57.755666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:42:57.766616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 85/165] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:57.767613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.904266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.942082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3edf68-ba66-40e3-9003-7b7c9aa4f203 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.989466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.990133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.990549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.990831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.991501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:57.991803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:57.993202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.002977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 86/166] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.006859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.064865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1788-df76-42d6-ba3b-dd5d2c5ee0c1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.068301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.122948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a926e-62de-45c8-9f12-18bbf22502c4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.142262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.143284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.143773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.144293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.145177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:58.145739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.147375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.161326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.161862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.162183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.162459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.163132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:58.163433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.165133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.166287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 81/167] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.168361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.177923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 87/168] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.177923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.301493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6de908-a4cf-4e50-99e0-c3035e83296d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.314263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.343837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.344495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.344793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.345140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.346798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.348677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.350969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 88/169] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.354099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.377796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54ee-83c3-490d-afa7-ec24e68e1dac None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.406975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.445441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.446387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.446866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.447288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] PATH_INFO: `/v3/users/c6e62ba856084972b01892225cf77493` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.454793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] RBAC: Authorizing `identity:get_user(user_id=c6e62ba856084972b01892225cf77493)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.456571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.458812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 82/170] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/users/c6e62ba856084972b01892225cf77493 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.459898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.478948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c6af-ca9a-4f93-bbf6-2c1045f12fc1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.520769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.521572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.522248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.522674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] PATH_INFO: `/v3/users/c2cca81b77904c7685856e765eb594be` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.524126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] RBAC: Authorizing `identity:get_user(user_id=c2cca81b77904c7685856e765eb594be)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.526127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.528567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 89/171] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/users/c2cca81b77904c7685856e765eb594be => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.529665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.587619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.653021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ac4b-5617-444d-9042-f512b3dd48c8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.704011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec660776-f9ed-4ff2-b359-591415f38a9a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.724917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.725958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.726451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.726951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.798323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.798960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.799246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.799523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.884487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.906705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.908935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.936150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.937873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:59.032251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 83/172] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:59.033470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.067896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.127626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 90/173] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:59.128498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.155412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.179131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539f2bf-78c1-49c1-b656-bbb51668e7b6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.244920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.246104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.246391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.246809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.282641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf2b6b-d213-4b0f-9668-e41b1a4a61b5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.351695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e529b3-b69a-45ee-81f7-4892026e205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.352436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e529b3-b69a-45ee-81f7-4892026e205b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.352673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e529b3-b69a-45ee-81f7-4892026e205b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.352951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e529b3-b69a-45ee-81f7-4892026e205b None admin] PATH_INFO: `/v3/projects/efc34c1702f747d5829b764643d3201e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.354340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e529b3-b69a-45ee-81f7-4892026e205b None admin] RBAC: Authorizing `identity:get_project(project_id=efc34c1702f747d5829b764643d3201e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:59.357279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e529b3-b69a-45ee-81f7-4892026e205b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:59.358981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 91/174] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/projects/efc34c1702f747d5829b764643d3201e => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:59.360151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653320fc-5497-4741-a088-8d058aef49b3 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.360431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653320fc-5497-4741-a088-8d058aef49b3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.360704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653320fc-5497-4741-a088-8d058aef49b3 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.361931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 92/175] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:42:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:59.367639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653320fc-5497-4741-a088-8d058aef49b3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.387202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:59.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:59.389577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.557877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.559871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653320fc-5497-4741-a088-8d058aef49b3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.584490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 84/176] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:59.585700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.693913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b27d3-16f5-437b-a3f0-80518ac75371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.694591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b27d3-16f5-437b-a3f0-80518ac75371 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.694879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b27d3-16f5-437b-a3f0-80518ac75371 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.695162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b27d3-16f5-437b-a3f0-80518ac75371 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.699029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b27d3-16f5-437b-a3f0-80518ac75371 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:59.699754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314349e7-c791-4df3-92e6-5d3952e6c3b7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.700778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b27d3-16f5-437b-a3f0-80518ac75371 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:59.704382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 93/177] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:59.704382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.704382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.704382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.734700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:59.735944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:59.769444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.770098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.770383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.770654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.771392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=efc34c1702f747d5829b764643d3201e, user.id=c2cca81b77904c7685856e765eb594be) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:59.771699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:59.773356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:59.786961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 85/178] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/role_assignments?user.id=c2cca81b77904c7685856e765eb594be&scope.project.id=efc34c1702f747d5829b764643d3201e&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:59.787828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.852237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 94/179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:59.853405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.882446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.930292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.040975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef754e-2622-4bfb-bd97-a60a4d455071 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.077391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93083998-5d43-42fa-b7f0-2f22faff32e5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.094473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faf4e0-5c80-4e41-9647-1213dfab6bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.095240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf4e0-5c80-4e41-9647-1213dfab6bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.095525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf4e0-5c80-4e41-9647-1213dfab6bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.095799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf4e0-5c80-4e41-9647-1213dfab6bf7 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.096940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf4e0-5c80-4e41-9647-1213dfab6bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.098976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf4e0-5c80-4e41-9647-1213dfab6bf7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.100705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 86/180] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:00.102036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3408c-edfd-4439-8c61-b7b1d5ae29c8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.102404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3408c-edfd-4439-8c61-b7b1d5ae29c8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.102625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3408c-edfd-4439-8c61-b7b1d5ae29c8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.103983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 87/181] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:00.105181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24c110-79f4-457c-9502-8ce1174ca2c2 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.105466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24c110-79f4-457c-9502-8ce1174ca2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.105738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24c110-79f4-457c-9502-8ce1174ca2c2 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.111186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 88/182] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:00.111186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24c110-79f4-457c-9502-8ce1174ca2c2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.134032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.135414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.136597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.137093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.138116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=c6e62ba856084972b01892225cf77493) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:00.139163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.141410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed846c3-0862-4436-89fa-413f664cc9ea None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.150709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 95/183] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/role_assignments?user.id=c6e62ba856084972b01892225cf77493&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:00.152323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e88f-5830-4274-ac88-715c0d4cf57f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.152974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e88f-5830-4274-ac88-715c0d4cf57f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.153460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e88f-5830-4274-ac88-715c0d4cf57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.176391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1db7e88f-5830-4274-ac88-715c0d4cf57f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:00.178270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1db7e88f-5830-4274-ac88-715c0d4cf57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:00.272814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 96/184] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:00.274406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a884-dd3c-44bf-9527-40e75bc72146 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.274742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a884-dd3c-44bf-9527-40e75bc72146 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.275108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a884-dd3c-44bf-9527-40e75bc72146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.281165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24c110-79f4-457c-9502-8ce1174ca2c2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.291041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.301306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e790a884-dd3c-44bf-9527-40e75bc72146 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:00.302581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e790a884-dd3c-44bf-9527-40e75bc72146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:00.314427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.339173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.339935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.340294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.340647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.341489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] RBAC: Adding query filter params (role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=c2cca81b77904c7685856e765eb594be) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:00.341877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.343556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.350026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 89/185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/role_assignments?user.id=c2cca81b77904c7685856e765eb594be&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:00.350985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.406869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 97/186] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:00.408243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42fe7e-f354-4eb0-84d1-3610caef968e None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.408532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42fe7e-f354-4eb0-84d1-3610caef968e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.408808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42fe7e-f354-4eb0-84d1-3610caef968e None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.412213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 98/187] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:00.421074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.421559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.421851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.455139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:00.457581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:00.564606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca3d7e-8a92-43b4-ae4f-976f1d28f46e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.577805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 99/188] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:00.610251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.610905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.611190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.611471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.624089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:00.629870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:00.630779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:00.631681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 90/189] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:00.637634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.663190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.723002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.741250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.792066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b90d9-b4b5-4503-82d2-c68f03b3b782 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.830397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.831041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.831325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.831600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.832722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.834212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.837955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 91/190] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:00.837955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.840836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822e5b9-6158-4d67-8e7f-3c13be6420d0 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.880607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.881234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.881515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.881788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.882505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:00.882813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.884202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.893489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 100/191] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:00.894317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.954422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c5f30-42f5-4729-b843-5d417ad7d8db None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.964361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.989910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.990578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.990849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.991130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.000477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd216-0fbf-4d55-9736-42971b33ce06 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.003441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:01.008719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:01.009629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:01.010474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 92/192] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:01.011215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.030628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.039252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.039875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.040162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.040437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.041102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.043343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.115923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d580fe7-caaa-405d-af73-5ca9fd8fc479 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.151030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.151874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.152456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.152757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.160005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 101/193] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:00 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:01.161039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.164061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:01.171031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:01.175781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 93/194] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:01.176616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.221882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.294455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93d281-d213-497f-b296-a0abbe79e952 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.306937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a74d7-c67d-43b9-badd-6e9f904d762b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.340852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.341509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.341796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.342128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] PATH_INFO: `/v3/users/43998783c73f466c83ae820bcea2ae8c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.348070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.349389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.349389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.349389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.349922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.350250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.351637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.358700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=43998783c73f466c83ae820bcea2ae8c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.360212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.362320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 94/195] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:01.363069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 102/196] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/users/43998783c73f466c83ae820bcea2ae8c => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:01.363455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.368088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.419745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.488678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.500991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a2b1-8d08-4c5c-ba04-931a0dab6242 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.501832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce83d-3df7-4560-9e84-b03b54472ad5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.544668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.545327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.545626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.546034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.547252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.548126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.548609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.549071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.549756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.550098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.551488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.561219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 95/197] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:01.561219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.637595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.638909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.687064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.704831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02459d3e-b7bd-4b19-bd5b-beea6f4999ac None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.745299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.746063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.746546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.746829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] PATH_INFO: `/v3/users/43998783c73f466c83ae820bcea2ae8c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.748031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] RBAC: Authorizing `identity:get_user(user_id=43998783c73f466c83ae820bcea2ae8c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.750760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.752570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 96/198] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/users/43998783c73f466c83ae820bcea2ae8c => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:01.753179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.768562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 103/199] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:43:01.770180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.779807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.890523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9561ac-8f57-4d09-a928-f3c869a428fb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.899128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312d26-3c8c-46cd-bed6-d088204e2528 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.930015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.931084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.931375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.931654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] PATH_INFO: `/v3/users/5762bde7301740b685093f725e8969d8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.933278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] RBAC: Authorizing `identity:get_user(user_id=5762bde7301740b685093f725e8969d8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.935760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.937623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 97/200] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/users/5762bde7301740b685093f725e8969d8 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:01.938432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.940515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.941491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.942117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.942617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.956114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:02.024886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:02.026322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:02.065550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:02.082000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eadfee-34db-4d4c-929c-652a701005d0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:02.129313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:02.130353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:02.130863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:02.131602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:02.133421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:02.135420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:02.138247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 98/201] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:02.139671 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:02.173943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 104/202] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:02.173943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:02.234975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:02.287886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354feb5-443d-4776-b32d-1ae8c0da615d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:02.308279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a9b97-75ef-46e8-9f3e-dec4e4dd0b6f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:02.352362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:02.353129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:02.353538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:02.353917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:02.355505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=43998783c73f466c83ae820bcea2ae8c) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:02.355955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:02.358066 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:02.358272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:02.359047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:02.359488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:02.359906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:02.365932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 105/203] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:02 2022] GET /identity/v3/role_assignments?user.id=43998783c73f466c83ae820bcea2ae8c&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:02.366910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:02.422675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:02.475462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:02.477343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:02.633121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 99/204] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:02.634456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:02.693944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:02.696932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257ac0-a190-4194-b7ef-ca490ba4eed9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:02.706303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:02.741446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:02.742121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:02.742406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:02.742683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:02.743825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:02.745515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:02.747218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 106/205] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:02 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:02.753379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:02.780732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e20855-0f2c-473a-851f-9aafc9358075 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:02.850580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:02.851593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:02.852116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:02.852591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:02.855459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:02.857342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:02.860072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 100/206] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:02 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:02.872085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:02.981919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62847ab-d37d-451a-b55c-2aa5beed11d6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:03.058202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae92b-b846-4528-b6b8-39f869aaf2ce None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:03.086343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:03.108545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:03.110085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.111144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.111623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.113384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] RBAC: Adding query filter params (role.id=1de11773e89446bc96f058fca76ee658, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7, user.id=5762bde7301740b685093f725e8969d8) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:03.113946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:03.115693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40795f-381d-49db-b94c-e63eb66d4271 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:03.130312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 101/207] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:02 2022] GET /identity/v3/role_assignments?user.id=5762bde7301740b685093f725e8969d8&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=1de11773e89446bc96f058fca76ee658 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:03.138785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:03.139429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.139713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.139985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.140744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] RBAC: Adding query filter params (role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=43998783c73f466c83ae820bcea2ae8c) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:03.141056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:03.150891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046dfd-3eaa-4948-b8d9-b8e7ab4973d4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:03.172189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 107/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:02 2022] GET /identity/v3/role_assignments?user.id=43998783c73f466c83ae820bcea2ae8c&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 259 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:03.482645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1103c56-e77e-4ce3-8359-cdeb9ab3f577 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.482645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1103c56-e77e-4ce3-8359-cdeb9ab3f577 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.482645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1103c56-e77e-4ce3-8359-cdeb9ab3f577 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.482645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 102/209] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:03.489688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0b88a-256f-47b9-8af7-0f82a6ffcfc3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.490263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0b88a-256f-47b9-8af7-0f82a6ffcfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.490737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0b88a-256f-47b9-8af7-0f82a6ffcfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.510592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-30e0b88a-256f-47b9-8af7-0f82a6ffcfc3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:03.512657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-30e0b88a-256f-47b9-8af7-0f82a6ffcfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:03.546403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:03.587516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 108/210] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:03.780552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3969-2afa-49c5-ac57-7f9e37914cde None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.780745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3969-2afa-49c5-ac57-7f9e37914cde None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.781151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3969-2afa-49c5-ac57-7f9e37914cde None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.782915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 103/211] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:03.789387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.789964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.789964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.813936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:03.815480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:03.896219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 109/212] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:03.900503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3969-2afa-49c5-ac57-7f9e37914cde None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.011935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3969-2afa-49c5-ac57-7f9e37914cde None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.048288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.048851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.049131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.049403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.060788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:04.065860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:04.066758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:04.067564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:04.072071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 104/213] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:04.074089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.188954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.192980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:04.200975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8c35a-b540-455e-9982-ebefe708c98e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.249642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.249642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.249642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.249986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.250550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:04.250854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.252872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.264087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 110/214] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:04.269188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:04.270598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.331121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782e1d4-1856-4c75-a595-9c2d821dad03 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.372534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.373827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.374363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.374949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.376381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.378228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.380392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 105/215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:04.386128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.410812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41b53c-1c17-4445-828c-904fa34a88dd None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.453039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.454077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.454589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.455081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] PATH_INFO: `/v3/users/0fb587d9c91d4cb8a9cc9628fbe82781` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.470485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] RBAC: Authorizing `identity:get_user(user_id=0fb587d9c91d4cb8a9cc9628fbe82781)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.472580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.474861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 111/216] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:04 2022] GET /identity/v3/users/0fb587d9c91d4cb8a9cc9628fbe82781 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:04.480696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.504729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:04.520318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816f67-a9fd-4729-bab4-dd2c8043331e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.539154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:04.560818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9796c5-c59e-4076-b014-2855a2282abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.561867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9796c5-c59e-4076-b014-2855a2282abb None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.562383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9796c5-c59e-4076-b014-2855a2282abb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.562886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9796c5-c59e-4076-b014-2855a2282abb None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.563748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9796c5-c59e-4076-b014-2855a2282abb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.565595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9796c5-c59e-4076-b014-2855a2282abb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.605933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941c020-2fc5-40b0-8967-b73f95afa09c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.643927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.644454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.644740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.645029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.681976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 106/217] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:04 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:43:04.681976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad0618-5ca7-4090-8068-c302237c1b3f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.681976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad0618-5ca7-4090-8068-c302237c1b3f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.681976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad0618-5ca7-4090-8068-c302237c1b3f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.681976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 107/218] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:04.687382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.687863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.687983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.708763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:04.710044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:04.732852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.734913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.756337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:04.812859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 108/219] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:04.860706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 112/220] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:43:04.866424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.978731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa6daf-5b37-4e40-b20d-0e0e0277d2be None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.980896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:05.017677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:05.018400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.018776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.019500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.021137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:05.023038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:05.030018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 109/221] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:04 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:05.031684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:05.103736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:05.148233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dbac4-c80f-4b34-884b-6f36709ddf86 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:05.168750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:05.190819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:05.191619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.192026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.192441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.193291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=0fb587d9c91d4cb8a9cc9628fbe82781) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:05.193816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:05.195506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42c526-4be2-4192-9225-bf2887f7f27a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:05.206311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 113/222] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:05 2022] GET /identity/v3/role_assignments?user.id=0fb587d9c91d4cb8a9cc9628fbe82781&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:05.223389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af584fc1-d019-4d22-a9a2-6725b4bc03fd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:05.264587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:05.265248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.265520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.265794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.273362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aca74-aef7-4ea1-8250-098feac3f9c6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.273650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aca74-aef7-4ea1-8250-098feac3f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.273967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aca74-aef7-4ea1-8250-098feac3f9c6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.275531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 114/223] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:05.279399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:05.283597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.283883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.284162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:05.284549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:05.285581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:05.286894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 110/224] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:05.293775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:05.307028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:05.308696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:05.330821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:05.444683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800307e2-e6de-4b20-99a9-386788b14d24 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:05.446141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 115/225] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:05.524007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:05.525029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.525529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.526036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.527000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:05.527515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:05.529188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:05.542662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 111/226] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:05.548207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:05.704268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94607e7-6365-4016-a033-e5d0f53358d1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:05.778640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:05.786565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:05.821633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:05.822490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.823010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.824146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] PATH_INFO: `/v3/users/43998783c73f466c83ae820bcea2ae8c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:05.827566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] RBAC: Authorizing `identity:get_user(user_id=43998783c73f466c83ae820bcea2ae8c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:05.833284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:05.836238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 116/227] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:05 2022] GET /identity/v3/users/43998783c73f466c83ae820bcea2ae8c => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:05.843880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:05.920719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:05.931299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0401581-4474-4d10-8857-e4e655341cd4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:05.982255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:05.982902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:05.983183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:05.983462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.008504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:06.013754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:06.014718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:06.015516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:06.022947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 112/228] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:06.023873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:06.061000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49787f3b-a843-48fd-a647-9a1c121f5332 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:06.111531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:06.112182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.112460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.112729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.118615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:06.120617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:06.122624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 117/229] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:05 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:06.128359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:06.177066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf80a74-5634-4444-ab2e-bd4441f836f7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:06.199061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:06.240476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:06.241195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.241611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.242019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.242795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:06.243313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:06.244946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:06.258353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 113/230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:06.260382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:06.278909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:06.281160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82d423-8801-4cf6-8f7f-1a0cd1122dd9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:06.329118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:06.330278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.330773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.333964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871/users/43998783c73f466c83ae820bcea2ae8c/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.339190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] RBAC: Authorizing `identity:create_grant(project_id=898065c5be12472e9c4f62d314668871, user_id=43998783c73f466c83ae820bcea2ae8c, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:06.341207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:06.407296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12864c9-96e0-4adc-9d57-99b5f4955a2a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:06.460960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 118/231] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:06 2022] PUT /identity/v3/projects/898065c5be12472e9c4f62d314668871/users/43998783c73f466c83ae820bcea2ae8c/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:43:06.469080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:06.470415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.470977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.471576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] PATH_INFO: `/v3/users/43998783c73f466c83ae820bcea2ae8c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.475063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] RBAC: Authorizing `identity:get_user(user_id=43998783c73f466c83ae820bcea2ae8c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:06.477291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:06.481064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 114/232] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:06 2022] GET /identity/v3/users/43998783c73f466c83ae820bcea2ae8c => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:06.487059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:06.694929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76aea9-165e-4be0-b9f3-7c0cfb1837ce None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:06.705068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:06.738910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eae4f0-85dd-4a8e-b733-5385dbdd003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:06.739568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eae4f0-85dd-4a8e-b733-5385dbdd003e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.739851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eae4f0-85dd-4a8e-b733-5385dbdd003e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.740126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eae4f0-85dd-4a8e-b733-5385dbdd003e None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.742685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eae4f0-85dd-4a8e-b733-5385dbdd003e None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:06.744457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eae4f0-85dd-4a8e-b733-5385dbdd003e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:06.747015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 119/233] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:06 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:06.753374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:06.864941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:06.887545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fea4f-8a78-497f-a552-c1de591c662b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:06.936216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a04e7-132a-4341-9cb7-835ec3ef77a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:06.936909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a04e7-132a-4341-9cb7-835ec3ef77a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.937224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a04e7-132a-4341-9cb7-835ec3ef77a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.937530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a04e7-132a-4341-9cb7-835ec3ef77a8 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871/users/43998783c73f466c83ae820bcea2ae8c/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.940052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a04e7-132a-4341-9cb7-835ec3ef77a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=898065c5be12472e9c4f62d314668871, user_id=43998783c73f466c83ae820bcea2ae8c, role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:06.941565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a04e7-132a-4341-9cb7-835ec3ef77a8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:07.053635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 115/234] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:06 2022] PUT /identity/v3/projects/898065c5be12472e9c4f62d314668871/users/43998783c73f466c83ae820bcea2ae8c/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:07.155100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763b9dc-f6d5-4729-878c-86cb74820345 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.155100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763b9dc-f6d5-4729-878c-86cb74820345 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.155100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763b9dc-f6d5-4729-878c-86cb74820345 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.156159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 120/235] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:07.164229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb02d-9377-4656-a300-27e81416ba2a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.164735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb02d-9377-4656-a300-27e81416ba2a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.165212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb02d-9377-4656-a300-27e81416ba2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.202007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-06fbb02d-9377-4656-a300-27e81416ba2a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:07.204069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-06fbb02d-9377-4656-a300-27e81416ba2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:07.335108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 116/236] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:07.491233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eecf7-222b-44a2-a0e3-96b0dbece0a5 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.491603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eecf7-222b-44a2-a0e3-96b0dbece0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.491780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eecf7-222b-44a2-a0e3-96b0dbece0a5 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.493408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 121/237] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:07.499070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b10f71-615b-40c2-a494-a22e3f52f74d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.499361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b10f71-615b-40c2-a494-a22e3f52f74d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.499640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b10f71-615b-40c2-a494-a22e3f52f74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.527525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d9b10f71-615b-40c2-a494-a22e3f52f74d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:07.528743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d9b10f71-615b-40c2-a494-a22e3f52f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:07.633275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 117/238] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:07.725877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce935549-409a-40ce-8e7e-2d05e8404107 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.725877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce935549-409a-40ce-8e7e-2d05e8404107 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.725877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce935549-409a-40ce-8e7e-2d05e8404107 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.726735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 122/239] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:07.733021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.733310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.733589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.746237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:07.763219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:07.764445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:07.812916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce935549-409a-40ce-8e7e-2d05e8404107 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:07.891956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 118/240] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:07.997364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce935549-409a-40ce-8e7e-2d05e8404107 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.051947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.052603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.052885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.053162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.056677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.078926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:08.084033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:08.084912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:08.085697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:08.087776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 123/241] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:08.093214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.176227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05a31-a4b7-4f8d-888a-46e9a24457d6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.178334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:08.208708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:08.214618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.215542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.216723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.216972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.218354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:08.219895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:08.221670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 119/242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:08.226445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.274553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83579e8e-4b44-4c16-aad7-375e9b252040 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.316466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.317186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.317474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.317748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.318627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:08.318939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:08.320442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:08.330515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 124/243] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:08.331435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.370296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf33ee-90cb-406f-88fb-b2029a8cb0f5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.415797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.416442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.417318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.417631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.428993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:08.434729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:08.435628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:08.436426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 120/244] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:08.437072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.454895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:08.462903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2966a-be92-4f9d-be8b-f371a00a02d1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.510821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.511511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:08.512102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.512971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.513778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.514681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:08.517174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:08.561879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8f079-14e9-417d-b80f-9a153e5e8095 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.606297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.606944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.607230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.607510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] PATH_INFO: `/v3/users/1aba08b55c5f4b37bc17eeca6884199a` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.625841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] RBAC: Authorizing `identity:get_user(user_id=1aba08b55c5f4b37bc17eeca6884199a)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:08.627693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:08.629696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 121/245] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/users/1aba08b55c5f4b37bc17eeca6884199a => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:08.630719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.633416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 125/246] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:08 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:08.637138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.768349 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa3f8a-368a-4839-b5ad-02b12c93de5b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.791332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:08.814659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.815635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.816714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.817204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.818444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:08.818961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:08.820574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:08.833032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 122/247] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:08.840734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:08.875759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f95c0-f906-4acd-88ab-b429cc9f6a2c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:08.923093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:08.924346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:08.924840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:08.925321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:08.957161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:08.998657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a409-6ac2-4bc4-ab80-b13f9f918941 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:09.024210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:09.028644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:09.045816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:09.046594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.046826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.047100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] PATH_INFO: `/v3/users/0fb587d9c91d4cb8a9cc9628fbe82781` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.048293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] RBAC: Authorizing `identity:get_user(user_id=0fb587d9c91d4cb8a9cc9628fbe82781)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:09.050446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:09.053081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 123/248] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/users/0fb587d9c91d4cb8a9cc9628fbe82781 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:09.058309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:09.118247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:09.168011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 126/249] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:43:09.173574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:09.179835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bddd38-7c13-4ee3-b874-7ee67dcda76c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:09.219211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:09.219848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.220134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.220442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.223368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:09.223368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:09.226046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 124/250] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:09 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:09.230925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:09.247427 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:09.305567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee9f19-7b40-4c40-b4a2-670e53889b66 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:09.345276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2fd35-2e89-4211-9468-aeea2558353e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:09.345697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:09.346545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.346972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.347445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.348937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:09.350651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:09.352803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 127/251] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:09 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:09.363252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:09.364704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:09.390195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdb581-dd8b-42d4-9dcb-d89db2b20ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:09.391000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdb581-dd8b-42d4-9dcb-d89db2b20ee8 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.391351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdb581-dd8b-42d4-9dcb-d89db2b20ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.391692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdb581-dd8b-42d4-9dcb-d89db2b20ee8 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871/users/0fb587d9c91d4cb8a9cc9628fbe82781/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.394006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdb581-dd8b-42d4-9dcb-d89db2b20ee8 None admin] RBAC: Authorizing `identity:create_grant(project_id=898065c5be12472e9c4f62d314668871, user_id=0fb587d9c91d4cb8a9cc9628fbe82781, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:09.395517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdb581-dd8b-42d4-9dcb-d89db2b20ee8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:09.499242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b54e65-d2ae-4a2a-aa75-f5bd0d2f45fc None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:09.504361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 125/252] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:09 2022] PUT /identity/v3/projects/898065c5be12472e9c4f62d314668871/users/0fb587d9c91d4cb8a9cc9628fbe82781/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:09.505735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe67-ef6f-4b53-8795-46b373ce4ca5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.509246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe67-ef6f-4b53-8795-46b373ce4ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.509573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe67-ef6f-4b53-8795-46b373ce4ca5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.511674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 126/253] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:09.518382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.518682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.518943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.540941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:09.542920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:09.544157 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:09.568654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:09.570588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.571144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.571684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.572705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] RBAC: Adding query filter params (role.id=1de11773e89446bc96f058fca76ee658, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=1aba08b55c5f4b37bc17eeca6884199a) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:09.573361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:09.575457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607b86-5eba-4d06-88d0-0278444f7cf1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:09.588123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 128/254] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:09 2022] GET /identity/v3/role_assignments?user.id=1aba08b55c5f4b37bc17eeca6884199a&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=1de11773e89446bc96f058fca76ee658 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:09.634118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a744a-6959-4e6a-b587-907424f90c2b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.634549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a744a-6959-4e6a-b587-907424f90c2b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.634921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a744a-6959-4e6a-b587-907424f90c2b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.636603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 129/255] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:09.642798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.642980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.643353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.666764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:09.668135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:09.669084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 127/256] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:09.751670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 130/257] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:10.078830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.103475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.267965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e8b3-c042-4255-b7b5-cc851e556d9a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.292108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c417a0-b196-4500-91a8-1ec8d8c4de8e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.295540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.320915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.326605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.327469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.327875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.328286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.342743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.343616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.344041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.344475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.355208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:10.362847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:10.363939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:10.364905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:10.367686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 131/258] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:10.371572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:10.375554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:10.388824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:10.389980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:10.391020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:10.393371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 128/259] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:10.399154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.517800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992d033-8b1d-43d3-9137-93328368e4ab None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.599476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.600033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.600352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.600627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.605921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:10.605921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.606141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5c70e-81a7-4cc0-a6e3-2a49b552a6b1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.607187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.622408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 129/260] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:10.645981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.676374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.677057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.677340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.677621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.678365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:10.678671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.680125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.693570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 132/261] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:10.702652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.711346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.765716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.842158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf09876-b95d-43a7-ac52-d6b1cbfc7eca None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.845754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb84dd4-9994-4b2b-b108-8432fc75a2da None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.882027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.883245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.883674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.883958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] PATH_INFO: `/v3/users/43998783c73f466c83ae820bcea2ae8c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.885577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] RBAC: Authorizing `identity:get_user(user_id=43998783c73f466c83ae820bcea2ae8c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.887190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.888859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 133/262] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/users/43998783c73f466c83ae820bcea2ae8c => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:10.894432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.927202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.927861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.928148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.928431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] PATH_INFO: `/v3/users/43998783c73f466c83ae820bcea2ae8c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.934711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] RBAC: Authorizing `identity:get_user(user_id=43998783c73f466c83ae820bcea2ae8c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.936187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.938216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 130/263] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/users/43998783c73f466c83ae820bcea2ae8c => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:10.959711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:11.024031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7119ba-ad0f-4589-9afe-58eaa6f63bd7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:11.027793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:11.068720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:11.069363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:11.069646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:11.069946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:11.157621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:11.159448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:11.222985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:11.239959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe537e1a-3d19-449f-8d73-e9bb229d067b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:11.280963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 134/264] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:43:11.286553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:11.298974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:11.383253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:11.383945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:11.384247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:11.384529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:11.402962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e4f3b-d546-42c4-815f-6e91944626da None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:11.442820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:11.443387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:11.443675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:11.443997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:11.445137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:11.446794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:11.449774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 135/265] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:11 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:11.456482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:11.543341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:11.581947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b865b-7915-4d4a-b76b-cd928fda3d03 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:11.603561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:11.610661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:11.615989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:11.616614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:11.616897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:11.617174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:11.617931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=43998783c73f466c83ae820bcea2ae8c) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:11.618396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:11.620801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:11.627865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 136/266] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:11 2022] GET /identity/v3/role_assignments?user.id=43998783c73f466c83ae820bcea2ae8c&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:11.752386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:11.765672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 131/267] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 806 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:11.771208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:11.926712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cdcc7-84fb-4d0d-a6a9-fc2c6e933595 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:11.965580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e94f2-9a71-4dd3-aad9-f2a3b06e3e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:11.966280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e94f2-9a71-4dd3-aad9-f2a3b06e3e34 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:11.966566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e94f2-9a71-4dd3-aad9-f2a3b06e3e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:11.966845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e94f2-9a71-4dd3-aad9-f2a3b06e3e34 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:11.968015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e94f2-9a71-4dd3-aad9-f2a3b06e3e34 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:11.969484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e94f2-9a71-4dd3-aad9-f2a3b06e3e34 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:11.971485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 137/268] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:11 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:11.977445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:12.131326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd3a70-fbb3-4ec3-94f9-f7439264dc86 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:12.177871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:12.178523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.178809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.179081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.179826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] RBAC: Adding query filter params (role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=43998783c73f466c83ae820bcea2ae8c) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:12.180139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:12.181615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffe00a-9c5f-4aa0-bcc1-aef432065135 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:12.194429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 132/269] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:11 2022] GET /identity/v3/role_assignments?user.id=43998783c73f466c83ae820bcea2ae8c&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:12.294169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb5fc0-4167-4056-a4a8-25b741647b48 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.294719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb5fc0-4167-4056-a4a8-25b741647b48 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.294719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb5fc0-4167-4056-a4a8-25b741647b48 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.295276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a47827-a089-48b0-9e87-0746a962dd55 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.295567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a47827-a089-48b0-9e87-0746a962dd55 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.295846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a47827-a089-48b0-9e87-0746a962dd55 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.296406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 138/270] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:12.297458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 133/271] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:12.303122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad55f-6c06-4252-bb09-44461ddcc377 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.303412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad55f-6c06-4252-bb09-44461ddcc377 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.303691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad55f-6c06-4252-bb09-44461ddcc377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.309616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7077505-8d9d-449a-b9f5-661a35a7a8a8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.310313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7077505-8d9d-449a-b9f5-661a35a7a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.310832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7077505-8d9d-449a-b9f5-661a35a7a8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.316938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:12.334138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a7077505-8d9d-449a-b9f5-661a35a7a8a8 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:12.335883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a7077505-8d9d-449a-b9f5-661a35a7a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:12.339323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4efad55f-6c06-4252-bb09-44461ddcc377 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:12.340572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4efad55f-6c06-4252-bb09-44461ddcc377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:12.449072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:12.465674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 134/272] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:12.468286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 139/273] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:12.470070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b84c8-9e6f-4d1a-aba6-88f94cffb43c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.470610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b84c8-9e6f-4d1a-aba6-88f94cffb43c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.471082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b84c8-9e6f-4d1a-aba6-88f94cffb43c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.472894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 135/274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:12.483740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.484387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.484887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.505408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:12.507044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:12.589384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 140/275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:12.765640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b84c8-9e6f-4d1a-aba6-88f94cffb43c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:12.777097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:12.877027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:12.906245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b84c8-9e6f-4d1a-aba6-88f94cffb43c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:12.920230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69812f0-845e-4195-9b87-ec9cf8e524aa None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:12.940303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a6a52-2cb3-4d4c-883c-b0fb673e372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:12.940938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6a52-2cb3-4d4c-883c-b0fb673e372e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.941215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6a52-2cb3-4d4c-883c-b0fb673e372e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.941484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6a52-2cb3-4d4c-883c-b0fb673e372e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.951998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a6a52-2cb3-4d4c-883c-b0fb673e372e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:12.955060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:12.955689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.955969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.956242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-301a6a52-2cb3-4d4c-883c-b0fb673e372e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:12.956832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:12.957743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:12.958604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 136/276] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:12.959240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935c565-1bcd-4fca-afbd-01fce6b6a025 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.959538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935c565-1bcd-4fca-afbd-01fce6b6a025 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.959772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935c565-1bcd-4fca-afbd-01fce6b6a025 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.960916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 137/277] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:12.963048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935c565-1bcd-4fca-afbd-01fce6b6a025 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:12.972004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:12.976985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:12.978077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:12.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 141/278] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:12.980086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.022095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.077086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935c565-1bcd-4fca-afbd-01fce6b6a025 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.090412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837790a-0772-42a5-b9b2-20d0d316d12b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.115238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.121198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39100758-8bb2-46d2-9004-339152eb0776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.121842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39100758-8bb2-46d2-9004-339152eb0776 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.122156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39100758-8bb2-46d2-9004-339152eb0776 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.122433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39100758-8bb2-46d2-9004-339152eb0776 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.123820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39100758-8bb2-46d2-9004-339152eb0776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.125886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39100758-8bb2-46d2-9004-339152eb0776 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.126892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 138/279] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.127016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.127602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.128042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.128201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.128291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.128475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.128749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.128883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:13.129190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.130617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.140890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 142/280] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.141758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.148741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:13.149983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:13.225832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 139/281] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:13.228488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.253872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2666f24-9567-49c3-a83f-a9610a160759 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.285944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.289121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.289660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.290037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.290217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.290895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:13.291783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.293365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.305941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 143/282] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.305941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.349471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe0044-d9ea-4238-8d9a-7f85ec45aea6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.375696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.390759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.391379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.391665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.392057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.392889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.394380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.430849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419429-1558-4f79-90c0-c5bf5a4fbb7f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.465640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.466306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.466588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.466865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] PATH_INFO: `/v3/users/36f5ee2a2f9b431491eb8942fcf0ec10` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.481349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] RBAC: Authorizing `identity:get_user(user_id=36f5ee2a2f9b431491eb8942fcf0ec10)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.482886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.484556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 144/283] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/users/36f5ee2a2f9b431491eb8942fcf0ec10 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.485318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.513702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 140/284] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:13 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:13.514601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.610083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2174a4d-ff25-4822-824f-5c545d52d927 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.620082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.647732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c27d2-0c76-4c07-968c-5bb16ad84950 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.649158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.649811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.650122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.651248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] PATH_INFO: `/v3/users/0fb587d9c91d4cb8a9cc9628fbe82781` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.651396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.652275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] RBAC: Authorizing `identity:get_user(user_id=0fb587d9c91d4cb8a9cc9628fbe82781)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.653759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.656162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 145/285] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/users/0fb587d9c91d4cb8a9cc9628fbe82781 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.656814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.694872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.695498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.696567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.696844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.698289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, user.id=36f5ee2a2f9b431491eb8942fcf0ec10, scope.system=all) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:13.698839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.700250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.706270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 141/286] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/role_assignments?user.id=36f5ee2a2f9b431491eb8942fcf0ec10&scope.system=all&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.707190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.794139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826120-566b-4c63-b44a-26160ea7a53f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.844838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.845496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.845778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.846109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.852579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63759df-4b31-45b8-bfea-9b49653ad806 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.863765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:13.871246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:13.872389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:13.873663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 146/287] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:13.878565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.900778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.902431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.902431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.902431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.923985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.947592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.995485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.998739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:14.003036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aded88f-7183-41f2-a85c-57f7d8277651 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:14.098782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:14.099482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:14.099773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:14.100052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:14.100747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:14.101058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:14.102554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:14.121025 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 142/288] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:14.123678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 147/289] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:14.127290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:14.135106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:14.200222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:14.235736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6a11e-29ea-46d5-a03b-5646aa1fad8f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:14.271153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:14.271797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:14.272107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:14.272350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:14.273501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:14.275000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:14.277579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 143/290] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:14 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:14.283604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:14.328384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:14.340504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386df56-3be5-4117-8b13-76fedad4d3ea None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:14.394493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc571841-330c-460f-86e5-0a98e1f9b753 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:14.418669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:14.419340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:14.419630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:14.419915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] PATH_INFO: `/v3/users/1aba08b55c5f4b37bc17eeca6884199a` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:14.421544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] RBAC: Authorizing `identity:get_user(user_id=1aba08b55c5f4b37bc17eeca6884199a)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:14.423097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:14.425046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 148/291] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:14 2022] GET /identity/v3/users/1aba08b55c5f4b37bc17eeca6884199a => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:14.431005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:14.431179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:14.431619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:14.431957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:14.432283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:14.433100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=0fb587d9c91d4cb8a9cc9628fbe82781) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:14.433411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:14.434916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:14.438260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:14.441427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 144/292] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:14 2022] GET /identity/v3/role_assignments?user.id=0fb587d9c91d4cb8a9cc9628fbe82781&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:14.613231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939d71f-b8b7-4a50-9ced-cf03b8700e75 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:14.657629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65087c96-5fc1-4628-b5a3-59cdd5a5fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:14.658228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65087c96-5fc1-4628-b5a3-59cdd5a5fe35 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:14.658511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65087c96-5fc1-4628-b5a3-59cdd5a5fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:14.658788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65087c96-5fc1-4628-b5a3-59cdd5a5fe35 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:14.659934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65087c96-5fc1-4628-b5a3-59cdd5a5fe35 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:14.661381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65087c96-5fc1-4628-b5a3-59cdd5a5fe35 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:14.663424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 149/293] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:14 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:14.669681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:14.897075 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b8a3a-213e-4cf0-a249-cf890a7e6f59 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:14.934968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393d2d1-3e3d-43e2-93c5-55e7b6234ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:14.935543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393d2d1-3e3d-43e2-93c5-55e7b6234ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:14.935824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393d2d1-3e3d-43e2-93c5-55e7b6234ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:14.936100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393d2d1-3e3d-43e2-93c5-55e7b6234ba8 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871/users/1aba08b55c5f4b37bc17eeca6884199a/roles/1de11773e89446bc96f058fca76ee658` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:14.938061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393d2d1-3e3d-43e2-93c5-55e7b6234ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=898065c5be12472e9c4f62d314668871, user_id=1aba08b55c5f4b37bc17eeca6884199a, role_id=1de11773e89446bc96f058fca76ee658)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:14.939518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393d2d1-3e3d-43e2-93c5-55e7b6234ba8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:15.022337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 145/294] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:14 2022] PUT /identity/v3/projects/898065c5be12472e9c4f62d314668871/users/1aba08b55c5f4b37bc17eeca6884199a/roles/1de11773e89446bc96f058fca76ee658 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:43:15.936301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0308118-6127-4673-927b-455e694936d3 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:15.936806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0308118-6127-4673-927b-455e694936d3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:15.937046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0308118-6127-4673-927b-455e694936d3 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:15.938824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 150/295] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:15.944166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451d95b-d584-4db6-a40f-45c98a63cfc3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:15.944590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451d95b-d584-4db6-a40f-45c98a63cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:15.944916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451d95b-d584-4db6-a40f-45c98a63cfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:15.964872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e451d95b-d584-4db6-a40f-45c98a63cfc3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:15.966113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e451d95b-d584-4db6-a40f-45c98a63cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:16.026872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.040970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 146/296] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:16.135552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3530-2d98-4de8-9e21-f0f864ded119 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.135844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3530-2d98-4de8-9e21-f0f864ded119 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.136126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3530-2d98-4de8-9e21-f0f864ded119 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.137493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 151/297] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:16.143061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.143348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.143628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.146057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e275c28-de59-41a8-8726-9ba1088e28f8 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.146626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e275c28-de59-41a8-8726-9ba1088e28f8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.147131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e275c28-de59-41a8-8726-9ba1088e28f8 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.149073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 152/298] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:16.155758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.156557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.157044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.164514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:16.166467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:16.179203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:16.180390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:16.211147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.258178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 153/299] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:16.262669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 147/300] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:16.332361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.443276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14609e-e36b-45d3-9d7b-6d3782f1c8c6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.479504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.480218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.480497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.480773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.491524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:16.497257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:16.498462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:16.499241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:16.503522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 154/301] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:16.504980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.529200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.558110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.627786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048defc-8bee-425d-a1ef-9222faa92e7c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.645334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeaab96-f598-48d3-a366-be2cde634d85 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.650022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.665601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.666866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.667152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.667506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.668102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:16.668407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:16.669796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:16.679534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 148/302] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:16.680273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.690472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.694926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.694926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.694926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.703776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:16.708920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:16.709869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:16.710699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 155/303] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:16.712139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.799446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33305e7-c833-4a08-92a9-5f29f5026b23 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.806624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.829764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d6299-7688-401d-9337-1efaf266d0e6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.835926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.836597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.836885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.837165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.838955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:16.840391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:16.842008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 149/304] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:16.842800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.874882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.875525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.875807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.876081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] PATH_INFO: `/v3/users/3d93b5d26e5646cca6abec3071d51df2` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.892734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] RBAC: Authorizing `identity:get_user(user_id=3d93b5d26e5646cca6abec3071d51df2)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:16.894353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:16.901585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 156/305] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/users/3d93b5d26e5646cca6abec3071d51df2 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:16.901585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.907822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.971213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae2c7-364e-42ba-9fd6-ffc7e91f0392 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.008982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:17.010096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.010450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.010726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.011393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:17.011695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:17.013060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:17.020659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7fc1f-33eb-4c71-a421-d0a43a35280f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.021669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 150/306] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:17.022585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:17.062159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf1987-2f93-44a3-9c06-4bf84ebe8b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:17.062682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf1987-2f93-44a3-9c06-4bf84ebe8b90 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.063000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf1987-2f93-44a3-9c06-4bf84ebe8b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.063301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf1987-2f93-44a3-9c06-4bf84ebe8b90 None admin] PATH_INFO: `/v3/groups` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.063970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf1987-2f93-44a3-9c06-4bf84ebe8b90 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:17.065542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf1987-2f93-44a3-9c06-4bf84ebe8b90 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:17.125925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fa3d9-34f4-48ef-8fb0-0db2375b18cf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.140615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:17.161672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:17.162353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.162616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.162896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.163624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] RBAC: Adding query filter params (role.id=1de11773e89446bc96f058fca76ee658, user.id=3d93b5d26e5646cca6abec3071d51df2, scope.system=all) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:17.163940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:17.165376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088fa03-786a-4f1b-8cba-3620756f16e4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:17.170309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 151/307] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 13:43:17 2022] GET /identity/v3/role_assignments?user.id=3d93b5d26e5646cca6abec3071d51df2&scope.system=all&role.id=1de11773e89446bc96f058fca76ee658 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:17.171456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c076554-b0ac-4694-b379-f7699454db56 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.171742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c076554-b0ac-4694-b379-f7699454db56 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.172020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c076554-b0ac-4694-b379-f7699454db56 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.173316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 152/308] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:17.174082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c076554-b0ac-4694-b379-f7699454db56 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:17.232746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:17.237954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 157/309] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 1 13:43:16 2022] POST /identity/v3/groups => generated 229 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:17.243368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.243703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.244546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.274264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:17.276609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:17.312472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c076554-b0ac-4694-b379-f7699454db56 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.359690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:17.360340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.360803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.361089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] PATH_INFO: `/v3/users/36f5ee2a2f9b431491eb8942fcf0ec10` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.362355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] RBAC: Authorizing `identity:get_user(user_id=36f5ee2a2f9b431491eb8942fcf0ec10)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:17.363858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:17.365866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 153/310] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:17 2022] GET /identity/v3/users/36f5ee2a2f9b431491eb8942fcf0ec10 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:17.371608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:17.376847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 158/311] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:17.437503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:17.500960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd05206-d7e8-4f1a-ac2c-9d20f8d97f09 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.541786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:17.542827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.543284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.543857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] PATH_INFO: `/v3/system/users/36f5ee2a2f9b431491eb8942fcf0ec10/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.545561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=36f5ee2a2f9b431491eb8942fcf0ec10, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:17.547142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:17.656804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 154/312] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 1 13:43:17 2022] PUT /identity/v3/system/users/36f5ee2a2f9b431491eb8942fcf0ec10/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:43:17.662469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:17.793228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4ec6a-b02a-4cea-a3bb-b4bd6db2f159 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.827231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:17.832938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:17.833579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:17.833938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:17.834234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:17.846374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:17.851470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:17.852319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:17.853152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 159/313] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:17.859467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:17.969011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf744083-8d71-4040-accb-85fe9093c50b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:17.982506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:18.007559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:18.008194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.008479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.008759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.009436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:18.010395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:18.011790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:18.026268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 155/314] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:18.027753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:18.143042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf9e96-6672-4726-9b05-72cee254d28c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:18.187848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:18.188495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.188774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.189051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] PATH_INFO: `/v3/users/1aba08b55c5f4b37bc17eeca6884199a` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.190267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] RBAC: Authorizing `identity:get_user(user_id=1aba08b55c5f4b37bc17eeca6884199a)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:18.191721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:18.193690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 160/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:18 2022] GET /identity/v3/users/1aba08b55c5f4b37bc17eeca6884199a => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:18.201119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:18.309159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869f804-fb36-4541-af5c-53b8e6a8b1d0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:18.348568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:18.349253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.349540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.349871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.393371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:18.424203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:18.425686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:18.540592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 156/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:18.542330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:18.622943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:18.668031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716afa4b-31bc-4c59-ab3a-d45c93ca5878 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:18.706748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0acc7-85c2-4ad4-bb57-5a3076f9c666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:18.707401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0acc7-85c2-4ad4-bb57-5a3076f9c666 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.707682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0acc7-85c2-4ad4-bb57-5a3076f9c666 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.707959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0acc7-85c2-4ad4-bb57-5a3076f9c666 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.709127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0acc7-85c2-4ad4-bb57-5a3076f9c666 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:18.710687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0acc7-85c2-4ad4-bb57-5a3076f9c666 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:18.712706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 161/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:18 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:18.718655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:18.798964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:18.835676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241c995-7b5f-41d5-b48b-02cba919fb92 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:18.873408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:18.874067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.874354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.874632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.875394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] RBAC: Adding query filter params (role.id=1de11773e89446bc96f058fca76ee658, scope.project.id=898065c5be12472e9c4f62d314668871, user.id=1aba08b55c5f4b37bc17eeca6884199a) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:18.875707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:18.877174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2144e9-3916-4d1b-bb97-79527dc4b4b9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:18.890053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 157/318] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:18 2022] GET /identity/v3/role_assignments?user.id=1aba08b55c5f4b37bc17eeca6884199a&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=1de11773e89446bc96f058fca76ee658 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:19.208803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef1c3-6514-4b8f-8a70-922f06de9148 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.209361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef1c3-6514-4b8f-8a70-922f06de9148 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.209628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef1c3-6514-4b8f-8a70-922f06de9148 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.210877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 162/319] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:19.216390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1df20-ba18-4e0b-9917-695b936df352 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.216883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1df20-ba18-4e0b-9917-695b936df352 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.216980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1df20-ba18-4e0b-9917-695b936df352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.236068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-44c1df20-ba18-4e0b-9917-695b936df352 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:19.237246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-44c1df20-ba18-4e0b-9917-695b936df352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:19.310331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 158/320] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:19.403360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769a8d4-0af7-4744-95b8-bc042e0262af None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.403683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769a8d4-0af7-4744-95b8-bc042e0262af None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.403924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769a8d4-0af7-4744-95b8-bc042e0262af None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.405956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 163/321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:19.411369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59c856-2bd3-4782-87a3-32f0cdf8a0ae None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.411709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59c856-2bd3-4782-87a3-32f0cdf8a0ae None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.411983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59c856-2bd3-4782-87a3-32f0cdf8a0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.431887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4a59c856-2bd3-4782-87a3-32f0cdf8a0ae None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:19.433097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4a59c856-2bd3-4782-87a3-32f0cdf8a0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:19.498630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:19.507945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 159/322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:19.580363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769a8d4-0af7-4744-95b8-bc042e0262af None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:19.673592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052237eb-3541-45b3-ae77-5bac2ad9e32a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.673879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052237eb-3541-45b3-ae77-5bac2ad9e32a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.674187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052237eb-3541-45b3-ae77-5bac2ad9e32a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.675832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 160/323] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:19.680888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.681181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.681530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.700477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:19.701676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:19.703268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769a8d4-0af7-4744-95b8-bc042e0262af None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:19.735950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:19.749461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:19.750150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.750435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.750712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.770844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:19.773117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 161/324] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:19.777337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:19.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:19.787242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:19.788194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 164/325] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:19.793089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:19.888584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ce7e-3c65-40c7-b8f3-94faf2513db5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:19.903383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089b00d-10a4-476d-a544-a51752a490a0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:19.922095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:19.934264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:19.934916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.935198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.935488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.936594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:19.938060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:19.939796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 162/326] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:19.939905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:19.940444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.940756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.941033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.941778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:19.942107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:19.943472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:19.945090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:19.955138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 165/327] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:19.960868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:20.016373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:20.063272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486d01-592e-4b6f-bdfe-9d0d5aad8029 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.072726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46c105-c772-4d46-ba55-975cd2e270da None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.098119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.098733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.099015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.099287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] PATH_INFO: `/v3/groups` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.099910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:20.101331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:20.106612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.107267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.107563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.107808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] PATH_INFO: `/v3/users/3d93b5d26e5646cca6abec3071d51df2` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.108945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] RBAC: Authorizing `identity:get_user(user_id=3d93b5d26e5646cca6abec3071d51df2)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:20.110429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:20.112016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 166/328] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:19 2022] GET /identity/v3/users/3d93b5d26e5646cca6abec3071d51df2 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:20.113956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:20.199797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 163/329] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 1 13:43:19 2022] POST /identity/v3/groups => generated 226 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:43:20.200735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:20.228904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6fe1c-de41-48f7-b831-bfa76e7a11c6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.262691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:20.286727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:20.292347 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.293010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.293294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.293597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.307038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:20.312741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:20.315632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:20.316428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 167/330] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:20.320013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:20.362370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527311f3-aa58-4059-9512-0abbda863d31 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.406080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.406746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.407031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.407338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] PATH_INFO: `/v3/system/users/3d93b5d26e5646cca6abec3071d51df2/roles/1de11773e89446bc96f058fca76ee658` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.408794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3d93b5d26e5646cca6abec3071d51df2, role_id=1de11773e89446bc96f058fca76ee658)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:20.410201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:20.461567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bb0d-6a73-4c84-9fb5-f8984618801a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.473772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 164/331] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 1 13:43:20 2022] PUT /identity/v3/system/users/3d93b5d26e5646cca6abec3071d51df2/roles/1de11773e89446bc96f058fca76ee658 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:20.505120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.505804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.506327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.506448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.507089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:20.507393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:20.508951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:20.521101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 168/332] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:20.528096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:20.670870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c7e5-f19a-43cf-a516-2c92dd8324fc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.693547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:20.715521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd9870-4df7-4856-bcfb-1ed8137f9fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.716179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd9870-4df7-4856-bcfb-1ed8137f9fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.716467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd9870-4df7-4856-bcfb-1ed8137f9fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.716748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd9870-4df7-4856-bcfb-1ed8137f9fa5 None admin] PATH_INFO: `/v3/users/36f5ee2a2f9b431491eb8942fcf0ec10` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.717968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd9870-4df7-4856-bcfb-1ed8137f9fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=36f5ee2a2f9b431491eb8942fcf0ec10)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:20.719412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd9870-4df7-4856-bcfb-1ed8137f9fa5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:20.721455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 165/333] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:20 2022] GET /identity/v3/users/36f5ee2a2f9b431491eb8942fcf0ec10 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:20.729841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:20.776949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:20.861563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dfdca-a591-400f-b87d-c5f7cd984fb0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:20.903371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:20.903999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:20.904282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:20.904558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:20.905306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] RBAC: Adding query filter params (role.id=e2d854cafb8b4baab26d81a4c699e2b4, user.id=36f5ee2a2f9b431491eb8942fcf0ec10, scope.system=all) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:20.905618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:20.907190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c6367-4d20-46bc-94d0-162b124c7a9a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:20.924422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 169/334] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 13:43:20 2022] GET /identity/v3/role_assignments?user.id=36f5ee2a2f9b431491eb8942fcf0ec10&scope.system=all&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:22.511837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ba75-c432-4661-984f-70df81365517 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.512363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ba75-c432-4661-984f-70df81365517 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.512591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ba75-c432-4661-984f-70df81365517 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.514198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 166/335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:22.531802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c7d8-4340-4e30-9675-309307a061a6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.532301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c7d8-4340-4e30-9675-309307a061a6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.532375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c7d8-4340-4e30-9675-309307a061a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.556989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-84b3c7d8-4340-4e30-9675-309307a061a6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:22.558430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-84b3c7d8-4340-4e30-9675-309307a061a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:22.640477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f7667-0b3c-4ed2-a852-875ab3447568 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.640766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f7667-0b3c-4ed2-a852-875ab3447568 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.641294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f7667-0b3c-4ed2-a852-875ab3447568 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.644364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 167/336] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:22.649174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.649730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.650274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.650616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 170/337] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:22.687578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:22.689195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:22.716802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24549750-4e48-49a9-bde6-a6b700859466 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.717094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24549750-4e48-49a9-bde6-a6b700859466 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.717379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24549750-4e48-49a9-bde6-a6b700859466 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.718908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 171/338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:22.725316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.725607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.726088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.758441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:22.760292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:22.791741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:22.809245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 168/339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:22.863761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 172/340] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:23.198598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.328217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c9f42-0d8c-4be5-bdca-e1248dbbedb0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:23.349060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:23.371268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:23.372607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.373128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.373609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.389879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:23.395317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:23.396204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:23.396996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:23.399325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 169/341] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:23.407019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.419441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.537657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d368f3d-c2f2-4922-9ce3-c944f1555646 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:23.581492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:23.582185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.582467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.582742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.595652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:23.600421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:23.601227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:23.602017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 170/342] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:23.606163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.639117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a07eb-8ed8-4ca7-a128-fd9f4013c681 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:23.641540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:23.679049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:23.679673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.680252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.680706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.682418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:23.682892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:23.684472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c96b-5088-43cd-ba4f-ae4c2980ed9e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:23.693773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 173/343] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:23.699545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401bcb-8a20-495f-9444-21d1a5f832a6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.700006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401bcb-8a20-495f-9444-21d1a5f832a6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.700424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401bcb-8a20-495f-9444-21d1a5f832a6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.701939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 174/344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:23.702872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5401bcb-8a20-495f-9444-21d1a5f832a6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.716612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d54c14-6978-4763-aef5-6b73aeaea25c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:23.741940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:23.755755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f3fe2-613e-4ec9-9bb0-661285b8634c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:23.756644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3fe2-613e-4ec9-9bb0-661285b8634c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.757292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3fe2-613e-4ec9-9bb0-661285b8634c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.757769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3fe2-613e-4ec9-9bb0-661285b8634c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.770060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f3fe2-613e-4ec9-9bb0-661285b8634c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-216f3fe2-613e-4ec9-9bb0-661285b8634c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:23.775289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:23.776406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:23.777436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:23.779407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 171/345] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:23.780732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.781163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.781583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.801617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:23.803104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:23.823822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5401bcb-8a20-495f-9444-21d1a5f832a6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:23.862804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:23.863481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:23.863771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:23.864057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:23.864827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:23.865609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:23.867653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:23.880067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 172/346] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:23.881163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.887172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 175/347] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:23.888205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:23.911216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:23.995338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfa8c1-1c7d-4fc1-93f6-46b72062bece None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.022376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.034602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.037461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.037461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.037461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.058252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0054e74-3266-4bf5-8927-e270d729ca11 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.110499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.112028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:24.123536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.124177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.124458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.124734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.125421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:24.125728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.127157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:24.140611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.141516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 176/348] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:24.142416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.272796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 173/349] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:24.273683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.310742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81edd82-0cf2-43b7-939d-b60c9405b418 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.358845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.359983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.360419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.360844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] PATH_INFO: `/v3/users/3d93b5d26e5646cca6abec3071d51df2` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.363432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] RBAC: Authorizing `identity:get_user(user_id=3d93b5d26e5646cca6abec3071d51df2)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.365439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:24.368594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 177/350] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/users/3d93b5d26e5646cca6abec3071d51df2 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:24.370141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.388184 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.403760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37672281-85b3-4893-a3a4-2025133dfb3b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.436137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.450054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.451101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.451668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.452149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.518194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3270-cb0e-4bc0-a45a-f87382d4c421 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.547122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.549302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:24.572098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.572680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.572970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.573254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.574903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.576179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:24.582554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 178/351] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:24.583662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.695816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.701971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 174/352] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:24.701971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.727548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70435c20-e0d2-42cd-bc35-f4ad1042365b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.731520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.779386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.779654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.780094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.780371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.792127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:24.797518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:24.798663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:24.799665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:24.803308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 179/353] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:24.804244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.838445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22097292-b9da-4c51-80c3-74ccbe6ea734 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.886573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.887189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.887469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.887745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.888473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] RBAC: Adding query filter params (role.id=1de11773e89446bc96f058fca76ee658, user.id=3d93b5d26e5646cca6abec3071d51df2, scope.system=all) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:24.888778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.891044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b60765-2713-4d1d-8c9b-5772f13df52f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:24.907658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 175/354] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/role_assignments?user.id=3d93b5d26e5646cca6abec3071d51df2&scope.system=all&role.id=1de11773e89446bc96f058fca76ee658 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:24.909264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3030c-2e17-4e19-9b0c-0ed9eead342d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.909586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3030c-2e17-4e19-9b0c-0ed9eead342d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.909872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3030c-2e17-4e19-9b0c-0ed9eead342d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.911343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 176/355] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:24.912137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3030c-2e17-4e19-9b0c-0ed9eead342d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:24.934192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83690030-d285-4f5f-ba84-a30687b4dfd8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:24.985282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:24.986371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.986649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.986925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.997684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:24.999487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.006309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 180/356] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.007363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.014290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.045050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.075104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3030c-2e17-4e19-9b0c-0ed9eead342d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.130935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1645220-6691-4082-99c2-66614d05e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.131519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1645220-6691-4082-99c2-66614d05e4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.131808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1645220-6691-4082-99c2-66614d05e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.132084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1645220-6691-4082-99c2-66614d05e4c9 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.133356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1645220-6691-4082-99c2-66614d05e4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.135574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1645220-6691-4082-99c2-66614d05e4c9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.138347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 177/357] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:24 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.140211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065b264-2778-4dcd-aa41-39641ebab3d5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.140496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065b264-2778-4dcd-aa41-39641ebab3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.140774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065b264-2778-4dcd-aa41-39641ebab3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.167580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d065b264-2778-4dcd-aa41-39641ebab3d5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:25.168865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d065b264-2778-4dcd-aa41-39641ebab3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:25.176192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11834260-31e3-40c1-ba03-2c91c5c9e5e4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.228514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.229519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.229804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.230183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.230888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:25.231199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.232623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.242737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 181/358] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.244015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.265971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 178/359] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:25.267230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897f33-d4f3-4af2-9b62-dde688017d0e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.267516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897f33-d4f3-4af2-9b62-dde688017d0e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.267792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897f33-d4f3-4af2-9b62-dde688017d0e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.269494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 179/360] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:25.270137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c897f33-d4f3-4af2-9b62-dde688017d0e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.333256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.386551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.390114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f80fc-4235-43dd-8eb3-3a51fbf3439f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.423125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c897f33-d4f3-4af2-9b62-dde688017d0e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.432386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.433011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.433352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.433660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.434538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:25.434745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.436209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.445730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 182/361] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.450819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.468134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c86954-f002-4597-b1df-8ecc4682f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.468792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c86954-f002-4597-b1df-8ecc4682f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.469071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c86954-f002-4597-b1df-8ecc4682f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.469345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c86954-f002-4597-b1df-8ecc4682f3d6 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.471732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c86954-f002-4597-b1df-8ecc4682f3d6 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.473215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c86954-f002-4597-b1df-8ecc4682f3d6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.475180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 180/362] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.481050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.481391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.481615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.506894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:25.508600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:25.599856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 181/363] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:25.599856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.624981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eb68d-3d6d-4ab7-b266-a4644b2757b0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.661471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.673352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.674133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.674466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.675457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.708348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.743372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b389a-86fd-40ba-97b8-4f31930e13a6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.771688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.773296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.792455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.793351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.793700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.794448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.795203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:25.795520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.796965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.806226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 182/364] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.807106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.941679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb47762-ee57-4f75-98c2-eb03291b5195 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.967098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 183/365] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:25.972970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.990744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.991800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.992092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.992377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.007250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:26.013425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:26.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:26.015185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 183/366] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:26.016351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.016449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.022005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.132387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d8944-78db-423e-ad4f-15bb54eae4d0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.159226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156f2cb-a4ec-45fb-8921-6e27623e23c0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.179624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.180293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.180587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.181398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.204438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:26.211588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:26.213046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:26.215073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:26.217522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 184/367] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:26.218785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.225469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.226144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.226428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.226700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.229038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:26.230626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:26.237186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 184/368] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:26.238236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.331780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.350408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.367474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b10383-f350-41ab-ad59-b734daff18e9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.373803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310f425-77b7-4bf8-8612-e72a6b3f30ac None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.414198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.414404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.414846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.415157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.415957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:26.416933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:26.417162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.417635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.418220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.418336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:26.419266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:26.419774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:26.421386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:26.429943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 185/369] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:26.430829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.434224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 185/370] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:26.440910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.569450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8d2aa-fc96-407c-97a0-2fdff972177d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.571512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e74a8-308c-45de-b0ca-6cd7ed96f50e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.620995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.629919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.630554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.631067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.637729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.637729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.637943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.637943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.638068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.638068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:26.638068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:26.638068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 186/371] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:26.642751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.671706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.728709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:26.731424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:26.785168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc078-049e-480f-b46a-1e42129fa077 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:26.825303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:26.826311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:26.826816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:26.827292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:26.863143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 186/372] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:26.864125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:26.896907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.913534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:26.916320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:26.937785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:26.974684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c14a1-5270-422f-8233-ded0b8cbb626 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.014411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.015176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.015530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.016144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.016778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=898065c5be12472e9c4f62d314668871) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:27.017166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.018692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.028743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 187/373] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:27.028743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:27.053939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 187/374] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:27.060573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:27.146788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:27.155919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5aa7a-cf9d-4a2b-a8b7-1b9a36959535 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.200139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.200801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.201088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.201369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.202774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.204334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.206047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 188/375] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:27 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:27.210741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2775-18c8-4cdc-98b1-04a26f052824 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.211884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:27.233054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:27.253223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.254337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.254823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.255292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] PATH_INFO: `/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.256896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] RBAC: Authorizing `identity:get_project(project_id=1cbeb42d06b34679b1e34ad6fea416bc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.258919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.262931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 188/376] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:27 2022] GET /identity/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:27.284716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:27.367512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21214bd-7630-466f-8085-c22205f15e99 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.408262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.409138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.409561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.409997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.410729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996e5b2-ade7-41bf-a122-334e2ab242d0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.411489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.413282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.416076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 189/377] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:27 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:27.422732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:27.450551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.451350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.451732 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.452096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] PATH_INFO: `/v3/groups/b175d32b151e428aa200042747c95349` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.457781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] RBAC: Authorizing `identity:get_group(group_id=b175d32b151e428aa200042747c95349)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.459483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.461705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 189/378] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:27 2022] GET /identity/v3/groups/b175d32b151e428aa200042747c95349 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:27.465473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:27.474837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:27.543878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:27.551432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de7f25-7f6d-4d90-8f14-2561df6fcb4a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.600619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda78f75-5a7c-464d-87b3-4bcd6bf9dc5d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:27.601222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.601858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.602171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.602707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.603460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=898065c5be12472e9c4f62d314668871) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:27.603769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.605403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44386251-0844-4211-b17d-0a3f5bc30a75 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.613114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 190/379] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:27 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:27.644291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:27.645076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.645457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.645828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.646727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] RBAC: Adding query filter params (group.id=b175d32b151e428aa200042747c95349, role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=1cbeb42d06b34679b1e34ad6fea416bc) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:27.647110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:27.648712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e28a-acba-4aec-8028-7f799b82b40b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:27.659039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 190/380] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:27 2022] GET /identity/v3/role_assignments?group.id=b175d32b151e428aa200042747c95349&scope.project.id=1cbeb42d06b34679b1e34ad6fea416bc&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:27.940673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8410-5ff1-45f2-bcc2-46892e5b3d0e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.941164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8410-5ff1-45f2-bcc2-46892e5b3d0e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.941164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8410-5ff1-45f2-bcc2-46892e5b3d0e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.942761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 191/381] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:27.949776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d0291-cf41-4ac5-8c3b-b55c9e147f93 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.950470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d0291-cf41-4ac5-8c3b-b55c9e147f93 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.950470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d0291-cf41-4ac5-8c3b-b55c9e147f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.978667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5c5d0291-cf41-4ac5-8c3b-b55c9e147f93 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:27.979892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5c5d0291-cf41-4ac5-8c3b-b55c9e147f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:28.068136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 191/382] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:28.254097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259599a-6f95-495e-9127-08a13f862b5c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.254548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259599a-6f95-495e-9127-08a13f862b5c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.254897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259599a-6f95-495e-9127-08a13f862b5c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.257105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 192/383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:28.263193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.263478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.263754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.285333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:28.286626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:28.334381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:28.367867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 192/384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:28.549989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259599a-6f95-495e-9127-08a13f862b5c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.670280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:28.683560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259599a-6f95-495e-9127-08a13f862b5c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:28.730108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:28.730749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.731030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.731302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.743889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:28.749048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:28.749920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:28.751279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 193/385] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:28.756173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.818143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.871166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cda4f-ac8e-44e9-ab2c-c322147b6121 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:28.909945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:28.910905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.911402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.911869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.912764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:28.913265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:28.914861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:28.924798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 193/386] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:28.938230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.988348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b614e-a297-41fc-bcef-c1b3ae45bddf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.038663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:29.059877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.060529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.060813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.061095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.080859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:29.089246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:29.090219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:29.091104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:29.092414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 194/387] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:29.096978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.142451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda6ff3-8fbc-45a9-a26d-974f45317b22 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.162413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:29.211475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.212389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.212692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.212973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.215474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.216954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.219668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 194/388] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:29.227090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.335914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fbd5-deab-421c-807b-175fda00407d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.404599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.405242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.405526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.405800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.406525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:29.406896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.409353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.421380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f757d4-cab2-478c-aecd-141a2fb422de None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.426314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:29.432424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 195/389] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:29.433425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.499835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.500491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.500774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.501050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.503019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.504415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.510267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 195/390] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:29 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:29.518193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.598740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:29.654894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ed985-43b3-409b-8eb9-de44b1ae8cc3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.718449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef29c3-ed08-4380-bab7-6214e53eab1c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.725754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.726449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.726730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.727006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.731462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.732917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.734622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 196/391] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:29 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:29.740271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.793067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcca53-4652-47fb-ad6e-99ce270ae353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.793749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcca53-4652-47fb-ad6e-99ce270ae353 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.794915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcca53-4652-47fb-ad6e-99ce270ae353 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.795203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcca53-4652-47fb-ad6e-99ce270ae353 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/groups/403bcc53c798405283443ea6077cfddc/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.797944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcca53-4652-47fb-ad6e-99ce270ae353 None admin] RBAC: Authorizing `identity:create_grant(project_id=86ef03b588f047dc8dc8400f1ac175a7, group_id=403bcc53c798405283443ea6077cfddc, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.799407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcca53-4652-47fb-ad6e-99ce270ae353 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.884346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 196/392] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 13:43:29 2022] PUT /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/groups/403bcc53c798405283443ea6077cfddc/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:43:29.892460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75616163-2223-45da-925d-5ecb9cadeb26 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.929999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69e861-8fed-4dfe-9f27-cbcab2658526 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.930320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69e861-8fed-4dfe-9f27-cbcab2658526 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.930661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69e861-8fed-4dfe-9f27-cbcab2658526 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.932302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 197/393] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:29.945449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.946252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.946756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.947213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.952796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.952989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c244-09fa-47b5-b4d0-4bcd0b30cc51 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.952989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c244-09fa-47b5-b4d0-4bcd0b30cc51 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.952989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c244-09fa-47b5-b4d0-4bcd0b30cc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.953244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.953244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 197/394] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:29 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:29.958211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.980108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:29.987399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cf09c244-09fa-47b5-b4d0-4bcd0b30cc51 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:29.989155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf09c244-09fa-47b5-b4d0-4bcd0b30cc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:30.041217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:30.096121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fee088-5062-42e1-98fa-b1f8243c6d4e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:30.140578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 198/395] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:30.149768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202308e2-a95a-45a7-b056-e689620bd924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:30.150553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202308e2-a95a-45a7-b056-e689620bd924 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.150841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202308e2-a95a-45a7-b056-e689620bd924 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.151123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202308e2-a95a-45a7-b056-e689620bd924 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/groups/403bcc53c798405283443ea6077cfddc/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.153407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202308e2-a95a-45a7-b056-e689620bd924 None admin] RBAC: Authorizing `identity:create_grant(project_id=86ef03b588f047dc8dc8400f1ac175a7, group_id=403bcc53c798405283443ea6077cfddc, role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:30.155442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202308e2-a95a-45a7-b056-e689620bd924 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:30.160864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcebd2-1453-4efe-bc50-58dcc92d01e2 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.161427 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcebd2-1453-4efe-bc50-58dcc92d01e2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.161996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcebd2-1453-4efe-bc50-58dcc92d01e2 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.163935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 199/396] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:30.169647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1def0f-d8b8-4fab-8264-f37e116b933a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.170343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1def0f-d8b8-4fab-8264-f37e116b933a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.170757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1def0f-d8b8-4fab-8264-f37e116b933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.192047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ef1def0f-d8b8-4fab-8264-f37e116b933a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:30.193609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ef1def0f-d8b8-4fab-8264-f37e116b933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:30.234068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 198/397] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 13:43:29 2022] PUT /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7/groups/403bcc53c798405283443ea6077cfddc/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:43:30.288711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 200/398] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:30.442292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3fb17-d075-4dce-a9b1-a281c4cd34b8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.442477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3fb17-d075-4dce-a9b1-a281c4cd34b8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.442769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3fb17-d075-4dce-a9b1-a281c4cd34b8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.444415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 199/399] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:30.464616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.465281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.465795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.490482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:30.492036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:30.554656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3fb17-d075-4dce-a9b1-a281c4cd34b8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:30.579642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 201/400] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:30.612515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:30.646375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:30.740006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:30.797562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3fb17-d075-4dce-a9b1-a281c4cd34b8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:30.808727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d08e14-5411-4a25-9371-8f41d287dd07 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:30.856214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:30.856862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.857149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.857428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.866448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:30.867451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:30.868032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:30.868521 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:30.868634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:30.874062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:30.875021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:30.875850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 200/401] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:30.876551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:30.884078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:30.889443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:30.890499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:30.891377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:30.894105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 202/402] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:30.895099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:30.987412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04449f-07e5-4350-8de3-b21d057dea8c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:30.988924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.022859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de701736-d8fa-4cec-87f6-f13da5485906 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.026358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.027245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.027726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.028215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.030593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.041272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:31.046985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:31.052194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 201/403] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:31.052999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.056608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.057246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.057515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.057791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.058498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:31.059686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.060324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.069961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 203/404] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.070805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.169990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a457d-8a0d-42c1-bb23-21c212ef7298 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.183969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d7c9e-5ef3-4228-b0d8-277f4864db07 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.207797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.208473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.208761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.209111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.209791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:31.210126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.211524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.222709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.223336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.223621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.223899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.224575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:31.225140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 202/405] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.225255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.226146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.226740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.237652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 204/406] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.238647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.249907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.294295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.345463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067e76c-1c5c-421d-93cf-defc2c1bf7c2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.356795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d418cda-1f03-4a9b-8ac6-4aef2a9939e0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.384557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.385560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.386059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.386724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.388059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.389949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.392078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 203/407] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.393096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.394316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.394964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.395250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.395530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.396768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.398593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.405225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 205/408] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.405225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.483374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.511723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e37c1-c6cd-482b-bb3e-bc6629fe0814 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.515923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b1e1-5a1c-4ab7-8d09-391679f182c3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.544626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.549677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.550306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.550592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.550873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.552032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.553504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.556662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 206/409] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.560799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.560936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.560936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.560936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.560936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] PATH_INFO: `/v3/groups/b175d32b151e428aa200042747c95349` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.562114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] RBAC: Authorizing `identity:get_group(group_id=b175d32b151e428aa200042747c95349)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.563779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.565913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 204/410] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/groups/b175d32b151e428aa200042747c95349 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.567006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.687304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06fb6c-be67-4345-abe9-a4338d1a5271 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.691980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc1838-3dd3-4951-95db-2329ae6d99ef None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.725372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.726152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.726445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.726835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.728319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.730368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.731137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.731968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.732084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 207/411] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.736877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.737068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.737714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871/groups/403bcc53c798405283443ea6077cfddc/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.740499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] RBAC: Authorizing `identity:create_grant(project_id=898065c5be12472e9c4f62d314668871, group_id=403bcc53c798405283443ea6077cfddc, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.742187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.747540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.847539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b539e68-2d2d-46f8-86eb-8f0fa520eb4e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.886295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.886944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.887224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.887520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] PATH_INFO: `/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.888698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] RBAC: Authorizing `identity:get_project(project_id=1cbeb42d06b34679b1e34ad6fea416bc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.890165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.896879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 208/412] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.896879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.926156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 205/413] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 13:43:31 2022] PUT /identity/v3/projects/898065c5be12472e9c4f62d314668871/groups/403bcc53c798405283443ea6077cfddc/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:31.927142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:32.075845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:32.087113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:32.115769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50795bc-682a-496a-9d87-67c53398b055 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:32.163542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14c1b5-be08-4d4e-97b3-90f6faac0a9f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:32.166257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788440-4f3f-49d1-819c-f87ecddb600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:32.167799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788440-4f3f-49d1-819c-f87ecddb600d None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.168100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788440-4f3f-49d1-819c-f87ecddb600d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.168389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788440-4f3f-49d1-819c-f87ecddb600d None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871/groups/403bcc53c798405283443ea6077cfddc/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.170787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788440-4f3f-49d1-819c-f87ecddb600d None admin] RBAC: Authorizing `identity:create_grant(project_id=898065c5be12472e9c4f62d314668871, group_id=403bcc53c798405283443ea6077cfddc, role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:32.172307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788440-4f3f-49d1-819c-f87ecddb600d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:32.205559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35be2d-c839-41e6-9611-9c20729286c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:32.206264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35be2d-c839-41e6-9611-9c20729286c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.206552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35be2d-c839-41e6-9611-9c20729286c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.206835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35be2d-c839-41e6-9611-9c20729286c3 None admin] PATH_INFO: `/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc/groups/b175d32b151e428aa200042747c95349/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.208768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35be2d-c839-41e6-9611-9c20729286c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1cbeb42d06b34679b1e34ad6fea416bc, group_id=b175d32b151e428aa200042747c95349, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:32.210291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35be2d-c839-41e6-9611-9c20729286c3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:32.244900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 209/414] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 13:43:31 2022] PUT /identity/v3/projects/898065c5be12472e9c4f62d314668871/groups/403bcc53c798405283443ea6077cfddc/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:32.253790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847705b2-a493-4d08-8434-1c6a94deb4c3 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.253790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847705b2-a493-4d08-8434-1c6a94deb4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.253790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847705b2-a493-4d08-8434-1c6a94deb4c3 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.253790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 210/415] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:32.256012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fe61d-3e0a-43e1-8f51-eda22a7498ba None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.256514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fe61d-3e0a-43e1-8f51-eda22a7498ba None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.256957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fe61d-3e0a-43e1-8f51-eda22a7498ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.278394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a00fe61d-3e0a-43e1-8f51-eda22a7498ba None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:32.279978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a00fe61d-3e0a-43e1-8f51-eda22a7498ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:32.307518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 206/416] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 13:43:31 2022] PUT /identity/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc/groups/b175d32b151e428aa200042747c95349/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:32.390866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 211/417] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:32.474213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496dff-ae17-44d5-a03b-37f88b2ece2f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.474757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496dff-ae17-44d5-a03b-37f88b2ece2f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.475188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496dff-ae17-44d5-a03b-37f88b2ece2f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.476673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 207/418] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:32.483201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.483613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.483994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.506787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:32.507995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:32.612040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:32.613272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 212/419] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:32.720912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496dff-ae17-44d5-a03b-37f88b2ece2f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:32.850834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496dff-ae17-44d5-a03b-37f88b2ece2f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:32.871675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:32.895512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:32.896248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.896635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.896979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.907774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:32.912821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:32.914013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:32.915339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:32.916854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 208/420] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:32.922786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:32.991729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:33.030663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90e29-a380-4c4e-8797-fd71a0df2c70 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.067536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.068031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.068372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.068708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.069443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:33.069809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:33.071963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:33.081188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 213/421] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:33.091927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:33.145716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204687-6e15-4890-927d-d3511af202ef None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.186018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.187045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.187479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.187918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.201091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:33.206915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:33.208437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:33.209516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:33.212347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 209/422] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:33.218552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:33.242357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:33.271168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:33.298641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5abea1-b747-4c0c-b7e3-bc1134c3591f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.334864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.335489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.335773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.336053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.405844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7798799-1073-44eb-b38d-ad8f6194f765 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.440751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:33.442383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:33.476987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.478552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.479004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.479429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.480366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:33.480925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:33.482506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:33.504236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 210/423] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:33.514377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:33.516994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:33.560971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 214/424] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:33.566848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:33.682052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bf5ce-582e-41cc-b003-4ac4dd13c17d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.722616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.723281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.723562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.723837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.725050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:33.726563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:33.729203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 215/425] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:33 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:33.734895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:33.840169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:33.848324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:33.864955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9a86b-f2fa-473d-a3d3-a28d09f35e04 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.929660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6c20b-9ab8-422c-9eb0-3e0ae7d33b47 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:33.958627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.959530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.959964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.960404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.966699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:33.967353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:33.967632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:33.967909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:33.973955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:33.973955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:33.973955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 216/426] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:33 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:33.978045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:34.103234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:34.104923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:34.150971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:34.157311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ea0d4-6804-4f6a-a115-b3107e005e97 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:34.194890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:34.195511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.195826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.196110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.196870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:34.197307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:34.198788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:34.205069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 217/427] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:33 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:34.212725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:34.222595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 211/428] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:34.228879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:34.359808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229038-cf97-4bce-8933-b70d932d67ce None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:34.397664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c424bcb-9009-478b-b740-b092ab0efacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:34.398348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c424bcb-9009-478b-b740-b092ab0efacf None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.398625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c424bcb-9009-478b-b740-b092ab0efacf None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.399486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c424bcb-9009-478b-b740-b092ab0efacf None admin] PATH_INFO: `/v3/projects/86ef03b588f047dc8dc8400f1ac175a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.400952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c424bcb-9009-478b-b740-b092ab0efacf None admin] RBAC: Authorizing `identity:get_project(project_id=86ef03b588f047dc8dc8400f1ac175a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:34.402477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c424bcb-9009-478b-b740-b092ab0efacf None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:34.404594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 218/429] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:34 2022] GET /identity/v3/projects/86ef03b588f047dc8dc8400f1ac175a7 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:34.410067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:34.428426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6097c21-1608-4e40-958a-4b20f3c70e5b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.428705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6097c21-1608-4e40-958a-4b20f3c70e5b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.428984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6097c21-1608-4e40-958a-4b20f3c70e5b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.430997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 219/430] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:34.437124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da693b-30be-434f-b417-5118725b7621 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.437442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da693b-30be-434f-b417-5118725b7621 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.437675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da693b-30be-434f-b417-5118725b7621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.475034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-03da693b-30be-434f-b417-5118725b7621 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:34.476257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-03da693b-30be-434f-b417-5118725b7621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:34.518324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:34.579310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb171a-0583-4488-a7f4-aa4e0988950f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:34.585670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 220/431] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:34.620419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a2662-fe23-4100-a9a2-e4d538ff4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:34.620962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a2662-fe23-4100-a9a2-e4d538ff4e0c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.621247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a2662-fe23-4100-a9a2-e4d538ff4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.621825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a2662-fe23-4100-a9a2-e4d538ff4e0c None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.623112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a2662-fe23-4100-a9a2-e4d538ff4e0c None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:34.624550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a2662-fe23-4100-a9a2-e4d538ff4e0c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:34.626606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 212/432] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:34 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:34.632255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da693b-30be-434f-b417-5118725b7621 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:34.729978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5f90-0d92-4516-98f7-9d36aed7275b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.730325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5f90-0d92-4516-98f7-9d36aed7275b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.730523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5f90-0d92-4516-98f7-9d36aed7275b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.734014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 213/433] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:34.744114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.744438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da693b-30be-434f-b417-5118725b7621 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:34.744556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.744709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.771444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:34.772837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:34.782268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:34.782889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.783170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.783446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.784192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=86ef03b588f047dc8dc8400f1ac175a7) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:34.784579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:34.786357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a30cf-e45f-4a88-bb28-ce482af212c0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:34.794064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 221/434] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:34 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=86ef03b588f047dc8dc8400f1ac175a7&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:34.795371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8ada8-0676-4865-83cd-66a448c7ed93 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.795654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8ada8-0676-4865-83cd-66a448c7ed93 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.795925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8ada8-0676-4865-83cd-66a448c7ed93 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.797464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 222/435] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:34.804665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.805041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.805844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.835930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:34.838958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:34.866809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:34.873548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:34.886793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 214/436] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:34.926809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:34.937938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 223/437] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:35.041702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66a7a6-18f0-4fcd-8de4-10be2a077647 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.079860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.080543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.080885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.081161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.092311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:35.097302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:35.098211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:35.099012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 215/438] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:35.104564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.199844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.224430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426de9c-1a44-48e4-bed0-93b3bdf0e723 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.272551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:35.273960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.274797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.275078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.275351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.276029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:35.276334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:35.277721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:35.291871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 224/439] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:35.293683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.313722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:35.334943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66669de2-0e64-45e2-8616-bbfcf774e62c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.372264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.372906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.373180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.373469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.385205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:35.390558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:35.391613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:35.392380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 216/440] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:35.393050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.413769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ced981-24e0-4a31-aaa6-26a866db3b99 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.449285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.450021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.450241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.450509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.462439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:35.467808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:35.468758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:35.469544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 225/441] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:35.470220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.498476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a4bc-43b3-437b-ac34-c733f5410dee None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.512819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:35.534422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.535053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.535337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.535617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.550682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:35.576980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd548a73-3522-4324-8908-1940405b19dd None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.610598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:35.612055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:35.612921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.613534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.613815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.614117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.614790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] RBAC: Adding query filter params (name=member) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:35.615092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:35.616456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:35.625853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 226/442] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:35.626637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.716306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 217/443] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:35.717114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.734525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:35.734778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357cef5-d1e7-40a3-846c-58cb1dcaed1a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.770135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.770749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.771035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.771310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.771981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:35.772289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:35.773986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:35.776610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:35.784272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 227/444] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:35.785105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.825441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235ad6-f0a5-4fa0-9977-3669a44b3aeb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.862467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.863088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.863370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.863654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.893579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463eafb-a0da-45fd-a7a3-6c4271e976d5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:35.929793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:35.930494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:35.930767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:35.931037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:35.932156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:35.933597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:35.935249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 228/445] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:35.935422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:35.936015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:35.936885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:35.951224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.003202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.038616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b7e40-7efb-4238-b746-64b90d3b3d1f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.040373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 218/446] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:36.041655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.076205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.076826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.077109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.077382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.147623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e60d73-5590-409e-98ba-47c34781b977 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.161090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.162981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.166297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.188389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.189035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.189267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.189682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.191755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.192323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.193957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 219/447] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:36.194741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.235200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.273147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 229/448] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:36.274002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.307805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072ff0f-4f89-43bd-851e-8292dfba51e8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.343336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.343958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.344241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.344514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] PATH_INFO: `/v3/projects/898065c5be12472e9c4f62d314668871` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.345645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] RBAC: Authorizing `identity:get_project(project_id=898065c5be12472e9c4f62d314668871)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.347091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.348728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 220/449] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/projects/898065c5be12472e9c4f62d314668871 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:36.349498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.381758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084e8d-76b2-4a40-b8de-97fc8b52a3f2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.403241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.420121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.420735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.421015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.424282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.424282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=898065c5be12472e9c4f62d314668871) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:36.424282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.424282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.430168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 230/450] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:36.431661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.468988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021be076-70d6-454b-b16f-b358ccf63cec None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.470286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.514029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.514775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.515060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.515329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] PATH_INFO: `/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.516610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] RBAC: Authorizing `identity:get_project(project_id=1cbeb42d06b34679b1e34ad6fea416bc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.518149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.520153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 221/451] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/projects/1cbeb42d06b34679b1e34ad6fea416bc => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:36.526534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.564215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284717f-7636-406f-bfe8-32f87fa32c42 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.615372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.616029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.616313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.616591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] PATH_INFO: `/v3/groups/403bcc53c798405283443ea6077cfddc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.618506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] RBAC: Authorizing `identity:get_group(group_id=403bcc53c798405283443ea6077cfddc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.619993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.622145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 231/452] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/groups/403bcc53c798405283443ea6077cfddc => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:36.627929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.661424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31044d7a-8d4f-4fd8-b1fe-48485c9dea70 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.678506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.698059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.698739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.699022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.699326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] PATH_INFO: `/v3/groups/b175d32b151e428aa200042747c95349` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.700468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] RBAC: Authorizing `identity:get_group(group_id=b175d32b151e428aa200042747c95349)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.701941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.703528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 222/453] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/groups/b175d32b151e428aa200042747c95349 => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:36.709445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.744192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:36.748680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2518a-02c5-477b-8826-eb3b78ce55b8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.787404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.787938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.788229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.788502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.789240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] RBAC: Adding query filter params (group.id=403bcc53c798405283443ea6077cfddc, role.id=e2d854cafb8b4baab26d81a4c699e2b4, scope.project.id=898065c5be12472e9c4f62d314668871) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:36.790424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.791250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374dcc5-5255-4248-99e7-5c45c9a7ce67 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.802186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 232/454] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/role_assignments?group.id=403bcc53c798405283443ea6077cfddc&scope.project.id=898065c5be12472e9c4f62d314668871&role.id=e2d854cafb8b4baab26d81a4c699e2b4 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:36.836083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8259-1926-4a38-acf0-e5327ec3b09a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:36.875563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:36.876193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.876508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.876824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.877599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] RBAC: Adding query filter params (group.id=b175d32b151e428aa200042747c95349, role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=1cbeb42d06b34679b1e34ad6fea416bc) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:36.877943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:36.879459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765e2de-c573-47c4-883b-5238f91ba716 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:36.887395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 223/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/role_assignments?group.id=b175d32b151e428aa200042747c95349&scope.project.id=1cbeb42d06b34679b1e34ad6fea416bc&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:39.632132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6716427-0dea-4f92-bd58-b0b73f7d5e41 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.633261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6716427-0dea-4f92-bd58-b0b73f7d5e41 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.633886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6716427-0dea-4f92-bd58-b0b73f7d5e41 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.635548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 233/456] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:39.641521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c103f0-0387-4e0e-99ad-1123d972a0a6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.642539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c103f0-0387-4e0e-99ad-1123d972a0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.643023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c103f0-0387-4e0e-99ad-1123d972a0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.670963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d1c103f0-0387-4e0e-99ad-1123d972a0a6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:39.672589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1c103f0-0387-4e0e-99ad-1123d972a0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:39.762306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:39.774365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 224/457] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:39.986822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb5520-5acc-477c-aa65-6ff9c7cf77db None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.987217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb5520-5acc-477c-aa65-6ff9c7cf77db None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.987393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb5520-5acc-477c-aa65-6ff9c7cf77db None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.988806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 234/458] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:40.005102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab909-0573-4f0c-b1ea-affa9d4e1078 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.005402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab909-0573-4f0c-b1ea-affa9d4e1078 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.005644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab909-0573-4f0c-b1ea-affa9d4e1078 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.007298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 235/459] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:40.007495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ba07-f518-4121-9c71-4e5ebac675e0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.007768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ba07-f518-4121-9c71-4e5ebac675e0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.008775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ba07-f518-4121-9c71-4e5ebac675e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.012831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.013103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.013375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.044524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:40.045739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:40.053807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4091ba07-f518-4121-9c71-4e5ebac675e0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:40.055136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4091ba07-f518-4121-9c71-4e5ebac675e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:40.178180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 236/460] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:40.179168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:40.183541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 225/461] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:40.285072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:40.294817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8465be-8f47-4773-8ca7-40f53d3b7668 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:40.337950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:40.338657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.338948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.339227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.350662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:40.353098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:40.376369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:40.377328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:40.378194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:40.378194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:40.378194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:40.378194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 237/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:40.379452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b976-d5eb-4ba0-9de6-b4a60003dced None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.379748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b976-d5eb-4ba0-9de6-b4a60003dced None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.380095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b976-d5eb-4ba0-9de6-b4a60003dced None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.381532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 226/463] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:40.383738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:40.390236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.390576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.390856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.431296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:40.433473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:40.538292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185b242-34cc-4d72-87e8-09b72509e491 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:40.578896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:40.579518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.579798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.580065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.580830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:40.581222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:40.582661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:40.585500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 227/464] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:40.586302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:40.618646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 238/465] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:40.620664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:40.626939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:40.640290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:40.728589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e17ba4-542b-434a-b7eb-4344d3d0ab43 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:40.761228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb5451-01e8-4edd-b3ec-9c31aa34a849 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:40.771538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:40.772473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.772938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.773451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.785999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:40.787908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:40.807400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:40.808553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:40.809707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:40.809707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:40.809707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:40.809929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:40.809929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:40.809929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:40.809929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:40.812084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 228/466] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:40.813218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:40.824987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:40.827175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:40.845375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:40.846772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:40.847904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:40.847904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:40.847904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:40.850957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 239/467] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:40.852891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:40.923517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:40.946372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:40.958060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831be53-e211-4bfe-8b17-53f603251341 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:40.976981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cc61c-ca2b-498b-9b18-1aad1d71b972 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.000255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.001024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.001306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.001581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.002315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.004063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.015258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.015883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.016165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.016439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.017092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:41.017528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.019186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.032052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 240/468] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:41.032986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:41.112801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 229/469] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:40 2022] POST /identity/v3/users => generated 246 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:41.113478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:41.182057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c637-2aad-47a7-8d39-bdba759947fd None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.206830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:41.229192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.229830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.230142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.230417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.231087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:41.231397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.232824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.246808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 241/470] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:41.247653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:41.257617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95630d00-cb31-480b-8775-04cdd7c77a2d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.310297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.311508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.312104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.312502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.328328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.331233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.344213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-bdfa50c6-ff14-4525-b48b-389e52eda064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:41.354896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:41.355811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:41.356644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:41.356644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:41.356644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:41.356644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 230/471] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:41.356993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f52c2-5866-454e-aabd-bd652895834f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.357782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f52c2-5866-454e-aabd-bd652895834f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.357782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f52c2-5866-454e-aabd-bd652895834f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.358889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 231/472] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:41.364531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f52c2-5866-454e-aabd-bd652895834f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:41.396048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746bb7-834a-44e4-b5f3-d267786842a6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.437557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.438242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.438519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.438829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.439594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.441000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.485995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f52c2-5866-454e-aabd-bd652895834f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.531643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395a5f0-c390-41dc-92b8-8e62ba852dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.532264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395a5f0-c390-41dc-92b8-8e62ba852dd3 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.532547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395a5f0-c390-41dc-92b8-8e62ba852dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.532825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395a5f0-c390-41dc-92b8-8e62ba852dd3 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.533477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395a5f0-c390-41dc-92b8-8e62ba852dd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.534935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395a5f0-c390-41dc-92b8-8e62ba852dd3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.550171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 242/473] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:41 2022] POST /identity/v3/users => generated 248 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:41.550171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:41.610600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:41.647473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 232/474] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:41 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:41.650298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.655834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.655834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.690502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:41.697774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:41.741981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3097fa9-6896-4adc-b6ac-a1895c9b2d26 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.800219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.801024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.801194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.802635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.802747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:41.803201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:41.803548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:41.805888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:41.820990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 243/475] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:41.827202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:41.828518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 233/476] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:41.942524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:41.952355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76ebf0-077f-423a-894d-8cafa5616658 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:41.995727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:41.996967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.997398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.997817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.998641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:42.000496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:42.110969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.132587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 244/477] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:41 2022] POST /identity/v3/users => generated 249 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:42.287323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c62b3-74d8-4ba6-b69f-451a95057eba None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.343895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.344530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.344810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.345084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.345792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:42.347254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:42.394720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.sql.core [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 13:43:42.394720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 13:43:42.394720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [parameters: {'id': '25f585b982b9495f9efdc1a23a1e8fb4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 13:43:42.394720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81706) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Sep 01 13:43:42.415698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:43:42.416632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application yield resource Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise self.value Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application session.commit() Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 13:43:42.417648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.flush() Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application compat.raise_( Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 13:43:42.418548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:43:42.419463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:43:42.420257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [parameters: {'id': '25f585b982b9495f9efdc1a23a1e8fb4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:42.421029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 13:43:42.421825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:42.435326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 234/478] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 13:43:42 2022] POST /identity/v3/roles => generated 144 bytes in 307 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:42.435491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.551738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bd871-f7f1-47c3-a818-c1554c20c61d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.590208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ba4ee-a800-4c60-b0b0-48a5e1fea61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.591222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ba4ee-a800-4c60-b0b0-48a5e1fea61b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.591704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ba4ee-a800-4c60-b0b0-48a5e1fea61b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.592268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ba4ee-a800-4c60-b0b0-48a5e1fea61b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.605406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ba4ee-a800-4c60-b0b0-48a5e1fea61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-3d6ba4ee-a800-4c60-b0b0-48a5e1fea61b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:42.611574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:42.612481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:43:42.613256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:42.616024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 245/479] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:43:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:42.622294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.682124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:42.741385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2973e3-d4cf-4822-885d-cf1c009383d2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.860054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.861027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.861027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.861292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.861808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:42.862139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:42.863523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6a47a-f8f3-4d9b-8481-b3284d362490 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:42.878265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 235/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:43.421495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8596-5462-4856-b334-c94742d65ac9 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:43.422053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8596-5462-4856-b334-c94742d65ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:43.422053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8596-5462-4856-b334-c94742d65ac9 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:43.423680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 246/481] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:43.429293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a140-5674-49ec-b878-1f4063c5abfc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:43.430572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a140-5674-49ec-b878-1f4063c5abfc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:43.430858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a140-5674-49ec-b878-1f4063c5abfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:43.459444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2aa6a140-5674-49ec-b878-1f4063c5abfc None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:43.460644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2aa6a140-5674-49ec-b878-1f4063c5abfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:43.550497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 236/482] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:43.894181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8596-5462-4856-b334-c94742d65ac9 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:43.908061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:44.037041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8596-5462-4856-b334-c94742d65ac9 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:44.083640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2babbf-b613-4648-8e1c-0a71f01d4b62 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.084190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2babbf-b613-4648-8e1c-0a71f01d4b62 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.084190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2babbf-b613-4648-8e1c-0a71f01d4b62 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.085191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:44.086345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 237/483] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:44.086834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.087329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.087798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.092496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8caa9-13cf-4cd9-9727-57bdf0596805 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.092782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8caa9-13cf-4cd9-9727-57bdf0596805 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.093055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8caa9-13cf-4cd9-9727-57bdf0596805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.107183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:44.112056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:44.112923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:44.113709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:44.119315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 247/484] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:44.121469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:44.126200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bca8caa9-13cf-4cd9-9727-57bdf0596805 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:44.127374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bca8caa9-13cf-4cd9-9727-57bdf0596805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:44.191414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:44.239742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 238/485] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:44.268069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:44.310962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2e8bc-6668-4b6d-b771-8cb0d626dc72 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:44.318632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d3063-24e0-47be-8efb-0ba1c5788126 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.319655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d3063-24e0-47be-8efb-0ba1c5788126 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.320817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d3063-24e0-47be-8efb-0ba1c5788126 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.323269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 239/486] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:44.328536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.329073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.329548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.359021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:44.360985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:44.370193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:44.371071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.371354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.371631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.372306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:44.372612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:44.375468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:44.390316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 248/487] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:44.396009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:44.484711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 240/488] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:44.573862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e884-381b-4bb7-900b-33f63dfa17e1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:44.607002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:44.614259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:44.624544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:44.625384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.625566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.629134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.638058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:44.647078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:44.665167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:44.666198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:44.667011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:44.667011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:44.667011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:44.674920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 249/489] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:44.680491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:44.717853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00acdad-a75d-437a-93d6-313033d66f21 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:44.723948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:44.834461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:44.835027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.835310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.836115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.847755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:44.852985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:44.853886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:44.854717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 241/490] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:44.855516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:44.882214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40e736-7e02-41d4-b7bf-1e60c0fbe650 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:44.919779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:44.920395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:44.920674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:44.920947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:44.921599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:44.921923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:44.923321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:44.936383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 250/491] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:44.937218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:44.971817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710c6c2-e110-4dc0-b565-43e6bcad0db4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:44.988775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.008722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16109ba8-b0a3-4a91-9323-2b15e4662fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.009358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16109ba8-b0a3-4a91-9323-2b15e4662fed None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.009642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16109ba8-b0a3-4a91-9323-2b15e4662fed None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.009941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16109ba8-b0a3-4a91-9323-2b15e4662fed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.021142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16109ba8-b0a3-4a91-9323-2b15e4662fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-16109ba8-b0a3-4a91-9323-2b15e4662fed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:45.026689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:45.027584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:45.028381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 242/492] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:45.029432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b7f4d-f3fe-41ee-9382-9c42c4b57e19 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.029432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b7f4d-f3fe-41ee-9382-9c42c4b57e19 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.029563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b7f4d-f3fe-41ee-9382-9c42c4b57e19 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.033928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 243/493] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:45.033928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b7f4d-f3fe-41ee-9382-9c42c4b57e19 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.055424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.059887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ca5e-2a8d-4cc5-8a98-718a41fcf138 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.097684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.098241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.098522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.098796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.099467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:45.099771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.101199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.110981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 251/494] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:45.111259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.145772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b7f4d-f3fe-41ee-9382-9c42c4b57e19 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.184448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.185087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.185370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.185646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.193358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.194843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-831b95a7-ac9e-409f-a8f2-1e36a7d542da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:45.208774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:45.210011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:45.211550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:45.215485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 244/495] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:45.216786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.217076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.217357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.225681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f259bc8-f8f8-41bf-a4b3-c7fcac89a8d3 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.228399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.235114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:45.236312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:45.264355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.264966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.265253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.265532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.266341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:45.266757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.268168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.281989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 252/496] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:45.281989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.309069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 245/497] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:45.309945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.333111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.405851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26411a-f611-4f93-bbf4-d6f235d6af2b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.443350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.443984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.444268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.444543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.448861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147262f-672e-4e7e-b2ef-515623aa627c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.456137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.457590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:45.473100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:45.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:45.475135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:45.475135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:45.475135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:45.475135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 253/498] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:45.475575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.491128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.491777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.492106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.492474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.493294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:45.493605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.495067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec216-f710-4f9f-bb7e-bb186899df38 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.496030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.539358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 246/499] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:45.539358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d4b04-cae8-408c-8f0b-bf8daf0f673c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.539358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d4b04-cae8-408c-8f0b-bf8daf0f673c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.539358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d4b04-cae8-408c-8f0b-bf8daf0f673c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.539358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 247/500] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:45.539358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d4b04-cae8-408c-8f0b-bf8daf0f673c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.596208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8f5c-d7ea-4385-a833-aecb568eebab None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.618600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.630414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.631033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.631316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.631583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.642096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.643534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.654175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d4b04-cae8-408c-8f0b-bf8daf0f673c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:45.658682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:45.659594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:45.660440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:45.660440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:45.660440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:45.661015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 254/501] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:45.662219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.690217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.690835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.696078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.696078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.696078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:45.696078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.696078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073bec8-a0f1-474e-ad9b-1a75fd1eda86 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.706342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 248/502] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:45.707566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.707853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.708135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.727989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:45.733762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:45.734855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.767311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15876d3-bf4b-4740-9a1e-c378e70ad583 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.796839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 249/503] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:45.797853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.802561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.803174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.803486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.803766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.878211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.879698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.880383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.928554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480e566-e11d-46f5-bc00-a0259106647f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.955603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.966664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.967306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.969923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.969923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.978743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:45.983994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:45.984916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:45.985734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 250/504] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:45.986524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.999194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 255/505] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:45.999194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.100382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bbabd-7ce8-4075-9e0d-3272a42f3ae1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.100856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae63cec-d007-4609-ad03-e8ec6328d255 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.118730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.136170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.136454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.136791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.137029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.137125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.137305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.137394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.137575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.138258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:46.138565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.140350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.144558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.146063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.152188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 251/506] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.153069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:46.165384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:46.166363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 256/507] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:46.167422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.199085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.260002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89bc3-5a0c-4317-95c3-a15ec2e7bcb9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.268737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7016ec-dec5-429e-9198-fc01705dbee3 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.294558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.295161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.295447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.295734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.296400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:46.296710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.298120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.307249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 252/508] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.307690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.308082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.308406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.308689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.309010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.323246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.324714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:46.339485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:46.340529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:46.341774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:46.341774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:46.341774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:46.341774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 257/509] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:46.341774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.358529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.419480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02863364-7ff9-48ab-9479-121481e7a3fb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.425419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.443274 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f4c7-3c96-416e-aedb-9ab6b6bb78f4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.455675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.456251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.456534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.456811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.469455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.470988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.479567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.480199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.480482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.480758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:46.486605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:46.487503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:46.488367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:46.488367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:46.488367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:46.488367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 253/510] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:46.488600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.489843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.491340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:46.504611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:46.505708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 258/511] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:46.506624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.590691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.599500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d23ab7-5356-4701-87ed-3379121109dd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.611440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e66cc6-2e2e-4c81-8db5-7e016937ea63 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.636120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.636796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.637089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.637360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.638117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:46.638426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.639849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.646143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.646771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.647055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.647321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.656878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 254/512] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.657713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.658135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.659576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.661480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:46.675911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:46.676933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:46.677764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:46.677764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:46.677764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:46.677764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 259/513] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:46.678028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.767268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903d74-d7d1-467a-8280-8d9c8de16988 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.795397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c5bb-4928-4b83-8f1c-86ee3a0c7790 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.802370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.802985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.803266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.803544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.804254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:46.805222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.806852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.818183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 255/514] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.819002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.831581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.836443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.837058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.837340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.837613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.838292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:46.838594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.839982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.853966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 260/515] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.854800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.912320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.923222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b9035-a86a-4a36-a802-b3ccdfefd43c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.957725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.958378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.958660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.958754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1172f-d1c0-461c-a301-db555c76f1b6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.958933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.959657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:46.959961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.961942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.986959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 256/516] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.987810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.997079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.997688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.997994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.998270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.060358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.072505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.074962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.096662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816be2f7-542b-4e54-a6d4-5b84926eca7f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.130227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.131092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.131700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.131984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.132387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.133534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:47.133863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.135258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.148993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 257/517] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:47.149642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.180999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 261/518] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:47.186013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.256004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead1ae2-98f8-4623-99c8-da47ccd9c278 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.282832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.284508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45259882-6710-4c17-8627-fe039b890d61 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.290938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.291576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.291855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.292130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.299392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.300743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:47.313004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:47.314077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 258/519] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:47.314916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.321582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.322228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.322509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.322781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.323432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.324874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.417082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f636-1e13-4105-b81f-7b3a426acbc0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.449258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 262/520] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:47 2022] POST /identity/v3/users => generated 247 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:47.450069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.451541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.452170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.452445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.452717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.465488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.518108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.536189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.537852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.577878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88895d1-1e63-4a9b-8c24-887860687d0d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.619113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.619906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.620972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.620972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.632926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.634584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:47.647706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:47.649624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:47.650627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 263/521] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:47.651704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.662067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 259/522] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:47.663121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.734773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.765804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.772216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009e12b-d793-4c37-960d-d0f25e3de796 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.778838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04ffcc-e4b9-4eb2-b9ff-9afdedc34f93 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.812993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.813884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.814351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.814789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.816511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.817126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.817417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.817683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:47.818451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:47.818762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.820488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.827478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:47.829265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:47.841229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 260/523] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:47.842137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:47.846705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:47.847643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:47.848496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:47.848496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:47.848496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:47.853042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 264/524] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:47.853751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:47.955414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e9bed-631d-4892-bb86-ba330f533e0d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.959711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078d856-cd4b-4b69-9d65-9a71e86435a1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:47.984177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:47.997149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:47.997789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:47.998106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:47.998359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.000246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.000703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.000991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.001293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.002068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:48.002381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.003821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.011325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.012966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.022499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 261/525] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:48.023850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.032379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:48.034935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:48.035908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:48.036745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:48.036745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:48.036745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:48.036745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 265/526] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:48.036971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.138875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a237c-3dc9-4398-be18-bd8a01b56f17 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.145441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea2d71-133b-4af1-8ae2-2d997f2ef405 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.178674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.179593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.179884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.180164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.180893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=cf5c9c7caaf4451a9b1ca1a1095408d1) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:48.181213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.182673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.184354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.184967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.185249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.185523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.186215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:48.186525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.188013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.198245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 262/527] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/role_assignments?user.id=cf5c9c7caaf4451a9b1ca1a1095408d1&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:48.198245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.216698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 266/528] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:48.217727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.263026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.285840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.325334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ba816-8dcd-423c-abd1-2c479ff6b728 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.366699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254a6d4-f7af-4a0c-bee8-4951179a111c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.373921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.373921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.373921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.373921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.408465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.409112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.409397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.409778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.410483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:48.410795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.412170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.426019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 267/529] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:48.427031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.455976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.457698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.535189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.546744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.550194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fce12-49fe-4123-ab6a-7abac6f9b0ff None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.595939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 263/530] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:48.595939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.600193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.601654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.602195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.602759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.612431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.615049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:48.635989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:48.637311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:48.638199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:48.640005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 268/531] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:48.640005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.724790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9599cc-b91c-4469-add9-2c5a853664cc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.763175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a4cde-232b-4240-ac47-cce1f59a98aa None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.764332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.765271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.765801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.766332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.774633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.776435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:48.790235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:48.791096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:48.791932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:48.794572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 264/532] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:48.795680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.799240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.799875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.800153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.800424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.813097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.814635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:48.829195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:48.830143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:48.831602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:48.831602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:48.831602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:48.834177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.835016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 269/533] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:48.836211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.845476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.914215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f068a1-6cf0-426e-b316-353d683f35e6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.942643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee0fb-88bf-4d15-8330-f893312b6081 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.953960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.955070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.955610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.956079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.957050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:48.957646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.959339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:48.973186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 265/534] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:48.974589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.980547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.980547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.980777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.981004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.981718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:48.982056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:48.983605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.002890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 270/535] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.003753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.072939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.081730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.092264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafba4e-13db-46c4-b053-c48158670c95 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.119454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f033cf-8c1c-4712-a67c-708e23114d92 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.134402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.135024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.135335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.135753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.137001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.137583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.140189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.157913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 266/536] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.159166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.163319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.164781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.165267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.165762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.166723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=4637080dea3a4349b53a3097f4610465) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.167249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.168898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.175900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 271/537] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/role_assignments?user.id=4637080dea3a4349b53a3097f4610465&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.177187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.301796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad09c-efa2-41ea-ba0c-7e28eeddc956 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.346074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.346744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.348033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.348326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.353484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=b9f9ea2ca628427bb165e1da9c0d1c7e) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.353484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.353484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.358885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 267/538] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/role_assignments?user.id=b9f9ea2ca628427bb165e1da9c0d1c7e&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.394417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba319a3-113f-4038-8ade-e8e05289fda0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.447898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.454377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.455038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.455323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.455600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.468475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.470112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6f17611d-18c5-431d-bef2-512104c1bb43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:49.488256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:49.489174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:49.490059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:49.495965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 272/539] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:49.501378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.534366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.609203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c8f46-3415-4706-b569-3dcaf0ce9b06 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.609751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c8f46-3415-4706-b569-3dcaf0ce9b06 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.610278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c8f46-3415-4706-b569-3dcaf0ce9b06 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.611955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 273/540] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:49.619558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.620098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.620936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.639520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf68a5-5de1-46dc-99f8-d32c78fa2872 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.641833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:49.643700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:49.691556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.693176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.693464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.694099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.694462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.695297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.696781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.710291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 268/541] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.718920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.778480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 274/542] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:49.832749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.868702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10be3c8-6545-4b6b-9d69-5d57ae468b46 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.917446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.918095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.918383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.918657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.919403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=2eeb9414ed49428684b9028c18476bba) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.919720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.921166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.929986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 269/543] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/role_assignments?user.id=2eeb9414ed49428684b9028c18476bba&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:50.162651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:50.281623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b308-63fa-46aa-95a3-f9f759e96629 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:50.305432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:50.321744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0d81d-28b7-45fd-9826-8d31bd116d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:50.322875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0d81d-28b7-45fd-9826-8d31bd116d31 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:50.323356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0d81d-28b7-45fd-9826-8d31bd116d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:50.323839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0d81d-28b7-45fd-9826-8d31bd116d31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:50.336260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0d81d-28b7-45fd-9826-8d31bd116d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5af0d81d-28b7-45fd-9826-8d31bd116d31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:50.341412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:50.342368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:50.343145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:50.345710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 275/544] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:50.351440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:50.438287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed8367-6375-473f-be35-926ce9f5c98b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:50.438592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed8367-6375-473f-be35-926ce9f5c98b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:50.438786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed8367-6375-473f-be35-926ce9f5c98b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:50.440815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 276/545] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:50.447271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:50.447550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:50.447819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:50.479032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:50.480229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:50.499190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f058ec-0f73-4e8b-bb5c-7575f05ed9b1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:50.539625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:50.542099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:50.542395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:50.542675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:50.543351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:50.543658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:50.545034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:50.555772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:50.563737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 270/546] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:50.570857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:50.629171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 277/547] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:50.701448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b34d4f-33a8-490d-9ad2-2f4fe65bde33 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:50.738712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:50.739707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:50.740161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:50.740602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:50.752764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:50.754772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:50.771286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:50.772252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:50.774558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:50.774558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:50.774558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:50.777006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 271/548] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:50.783860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:50.941492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee299-2696-44f7-9c67-4b6e1931c408 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:50.953236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:50.978843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:50.979459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:50.979739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:50.980016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:50.980664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:50.980966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:50.982372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:50.995842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 278/549] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:51.000950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.047809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.082261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:51.176324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc135a7-b3e9-44a2-a5ef-1544b231b6a7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:51.243047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa050d-8fe6-47f4-8663-ae6bb39744ae None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:51.243252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:51.243689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.243977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.244256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.267150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:51.268708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:51.282557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003988d-b89e-41f5-a789-993082346855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:51.283206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003988d-b89e-41f5-a789-993082346855 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.283481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003988d-b89e-41f5-a789-993082346855 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.283733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003988d-b89e-41f5-a789-993082346855 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:51.290513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:51.292787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:51.295060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 272/550] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:51.300480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.302971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003988d-b89e-41f5-a789-993082346855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-8003988d-b89e-41f5-a789-993082346855 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:51.313606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:51.315861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:51.317849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:51.323405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 279/551] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:51.325354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003988d-b89e-41f5-a789-993082346855 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.382673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:51.501729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd09e4-d6ef-45c6-955d-694310dca524 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:51.503076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003988d-b89e-41f5-a789-993082346855 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:51.534160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:51.539827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:51.540359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:51.540564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.540722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.540952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.541123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.541375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.541563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.541729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:51.541911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:51.542447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:51.548902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:51.549078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab2c74-1f4d-4a36-b5fc-0f1abaa69936 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:51.549220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ba0a-f5da-41ae-8f7b-b44e33830bb1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:51.552130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 280/552] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:51.553468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478c596-3bc1-428a-8e4b-3464e393ab68 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.553761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478c596-3bc1-428a-8e4b-3464e393ab68 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.554725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478c596-3bc1-428a-8e4b-3464e393ab68 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.556334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 281/553] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:51.564913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 273/554] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 13:43:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:51.566845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478c596-3bc1-428a-8e4b-3464e393ab68 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.567649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.568279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.568658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.588454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:51.589776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:51.671141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 274/555] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:51.672473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.710551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:51.739652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478c596-3bc1-428a-8e4b-3464e393ab68 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:51.812466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:51.813241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.813396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.813552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.825531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:51.827100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:51.829743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:51.833740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960a6e4-51f1-457d-91df-a181f5e6e285 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-3ee6fced-a040-4855-bc50-df21658092e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:51.848002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:51.850341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:51.853192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:51.853192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:51.853192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:51.857952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 282/556] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:51.857952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8483-310a-41de-bb8d-6c79b2604261 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.857952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8483-310a-41de-bb8d-6c79b2604261 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.857952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8483-310a-41de-bb8d-6c79b2604261 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.862657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 283/557] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:51.863459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8483-310a-41de-bb8d-6c79b2604261 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:51.872459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:51.873066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:51.873346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:51.873622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:51.945183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:51.946706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:51.981639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8483-310a-41de-bb8d-6c79b2604261 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.018261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.018882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.019165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.019444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.020085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:52.020391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.021783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710c445-8ebb-4981-b65c-e5701108eba4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.034711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 284/558] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:52.036038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.036584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.036584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.046296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.053873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.058754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:52.060017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:52.062324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 275/559] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:52.063727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.133495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 285/560] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:52.134386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.188544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add9cdf-543c-4a15-897b-2e14f20f0aa2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.238170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.239578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.240301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.240790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.258260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:52.260029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723624d4-a43f-4cf9-b4ae-032a1a582c95 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:52.265237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:52.266224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:52.267066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:52.269869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 276/561] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:52.271358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.298180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.298983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.299295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.299701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.301671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.308556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.310134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:52.322424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:52.324647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 286/562] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:52.326126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.343321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.393527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b3948-5eed-4ccc-b446-94d367e9b33a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.429504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.430173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.430449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.430724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.433500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88aba-2357-46b9-8454-33fa1f847c4f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.441815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.443298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5b6c262a-40da-4e8c-a474-213939216b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:52.458526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:52.461008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 277/563] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751dac5-1e01-4f47-9fe5-fc39b8de29cb None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751dac5-1e01-4f47-9fe5-fc39b8de29cb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751dac5-1e01-4f47-9fe5-fc39b8de29cb None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.463621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 278/564] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:52.463826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751dac5-1e01-4f47-9fe5-fc39b8de29cb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.473961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.474572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.474855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.475129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.475835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:52.476143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.477514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.485755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 287/565] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:52.486572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.543862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.569529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751dac5-1e01-4f47-9fe5-fc39b8de29cb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.583305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.592797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0acbdc-30ca-463e-8924-4b81d52cbb7f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.605195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.605809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.606121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.606403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.607140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:52.607451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.608876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.626257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 279/566] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:52.627168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.628306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64e14e-933f-4f92-8daa-1d4d9adbd669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.629000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64e14e-933f-4f92-8daa-1d4d9adbd669 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.629350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64e14e-933f-4f92-8daa-1d4d9adbd669 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.629694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64e14e-933f-4f92-8daa-1d4d9adbd669 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.641078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64e14e-933f-4f92-8daa-1d4d9adbd669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-1a64e14e-933f-4f92-8daa-1d4d9adbd669 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:52.646486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:52.648084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:52.648939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 288/567] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:52.649767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.650044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.650327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.671612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:52.672851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:52.743083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca776a-93f6-4f7c-bfd3-83c227e6b434 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.747530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 289/568] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:52.748385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.779060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.779670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.779950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.780223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.780867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:52.781174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.782573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.794200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 280/569] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:52.795030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.807716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.828622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:52.867409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123026a0-a8d5-4de2-9186-052010238b99 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.906367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.906855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06665b22-ed2d-4034-b1e0-096d482767b2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:52.907001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.907277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.907544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.919369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.921202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:52.939653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:52.946307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:52.947345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:52.947345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:52.947345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:52.947345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 290/570] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:52.947345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:52.954763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:52.955534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:52.955916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:52.956309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:52.968814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:52.970450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:52.987352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:52.988530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:52.989572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:52.989572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:52.989572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:52.991116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 281/571] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:52.992166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.078428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.082681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c6097-ff23-436f-804a-2d4b73d36852 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.087723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.108396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724510-af3a-4247-8c05-a8c693e264d3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.119060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.119593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.119865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.120148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.121094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:53.121434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.122909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.131670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 291/572] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:53.132739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.144914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.145543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.145822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.146129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.153879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.155497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:53.168896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:53.170206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:53.171283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:53.174740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 282/573] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:53.175574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.255971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fcb49-da94-4d7f-b89f-29b945c8ff8b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.291556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe9bf2-5779-4066-abbf-dd5372b0efa3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.293245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.293863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.294157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.294438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.295087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:53.295382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.296749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.309495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 292/574] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:53.310471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.332894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.333923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.334565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.334841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.335694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.336338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:53.336644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.338212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.353169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 283/575] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:53.354183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.357051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.420056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300f914-dca2-4493-9372-57d4f2035d94 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.456163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.456803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.457083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.457353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.469485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:53.474482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:53.475692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:53.478973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 293/576] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:53.479796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce35996-4da5-4eed-9613-0a34d42b850f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.479893 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.527316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.528003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.528359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.528629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.540943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.543732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:53.561122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:53.562392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:53.563519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:53.563519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:53.563519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:53.563758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 284/577] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:53.564974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.591471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.613536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c546d9e-8ac7-4562-a94d-d41401b973da None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.631466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.657353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.657981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.658270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.658553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.659260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:53.659566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.661027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.673605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 294/578] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:53.674419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.690968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd67f7-e156-4cf5-905e-8d0454d64b98 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.729091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.729751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.730059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.730335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.738324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.739762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:53.754495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:53.755693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 285/579] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:53.756646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.823257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595d80-6b3a-47cb-ae73-7a4bd3dfa3e5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.859749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.860386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.860669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.860943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.868814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.870570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.870792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb114c-4816-4d90-a765-94ed9cd29965 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:53.873241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:53.883354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:53.885028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 295/580] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:53.886210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:53.893206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:53.913527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:53.914220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:53.914457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:53.914730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:53.915672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:53.915795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:53.917084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:53.925519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 286/581] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:53.926377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.000374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4bbb-edc5-4f51-be07-2a1dd222db14 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.031103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c89bef-458e-4f18-b74a-0f4ea3169083 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.034705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.035314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.035593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.035864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.036496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:54.036798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.038332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.049789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 296/582] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:54.050626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.067335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.067972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.068257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.068535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.069278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=f0506db33e0e47aaa47a9a3fd9bc2f98) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:54.070362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.071783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.077405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 287/583] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:53 2022] GET /identity/v3/role_assignments?user.id=f0506db33e0e47aaa47a9a3fd9bc2f98&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:54.078050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.124164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.133700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.172661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eefdca-61fe-44b3-9acd-7432c9e4d0c8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.213008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564b0a9-1b84-4bce-b283-044551cea9bb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.215019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.215659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.215943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.216221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.216982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:54.217295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.218810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.259269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.259861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.260108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.260538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.261089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:54.261403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.262858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.264060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 297/584] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:54.264980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.281646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 288/585] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:54.281646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.383576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6f73-71b9-4f03-9d60-3b05e7d6dc08 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.387101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.390218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62782272-c7b7-4f88-bf49-492e0f15a6f9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.406124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.425403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.426392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.426781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.427145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.428217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.429470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.429748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.430056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.437874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.439290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.439404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.441118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:54.454080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:54.455204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:43:54.456151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:54.456679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 289/586] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:54.457959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:54.459424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:54.460410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:54.461282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:54.461282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:54.461282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:54.462914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 298/587] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:54.463902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.568346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a47a2-1ae3-4be6-a3ff-6de55bd2814e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.581481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227ad3-f269-485a-be89-d21913a9b812 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.613467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.614756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.615248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.615738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.618447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.619300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.619733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.620010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/cf5c9c7caaf4451a9b1ca1a1095408d1/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.629476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.631217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.635833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=cf5c9c7caaf4451a9b1ca1a1095408d1, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.637272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:54.648437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:54.649821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:54.650818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:54.650818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:54.650818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:54.650818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 290/588] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:54.650818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.667099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.715545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 299/589] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:54 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/cf5c9c7caaf4451a9b1ca1a1095408d1/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:54.727283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.764707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9b180-481f-4a9c-8161-ab0d504b5ad4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.772829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.818795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.819813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.820081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.820388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.821108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:54.821647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.823155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.852928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 291/590] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:54.853824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.892916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45dc89-7b03-47fb-a387-38eae3f0f4eb None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:54.936295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:54.937299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:54.937787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:54.938418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:54.939276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:54.939784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:54.941385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:54.942193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:54.960700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 300/591] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:54.963127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:54.991651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7562-567c-4d54-9403-be90c1f50502 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.032737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.033774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.034083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.034361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.035279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:55.035590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.037208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.058013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 292/592] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:55.059339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.065527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.093604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98264bc0-85a1-427d-887c-48285a7a8ef5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.136190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.137211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.137723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.138219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.150592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.152453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:55.169222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:55.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:55.171381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:55.171381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:55.171381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:55.177382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55b23-8cd5-4ab1-871f-89e59ae48d02 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.177632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 301/593] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:55.179317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.209732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.217986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.218641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.218906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.221222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.227554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.229101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:55.242505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:55.243429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:43:55.244362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:55.248788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 293/594] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:55.249661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.310422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd588-2002-494c-ab7b-551b2d7ef502 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.350675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.358505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.359638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.360118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.364456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.364753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1159a9e-9fb5-43d3-9355-0029ae6561f2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.370384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.372441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:55.390723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:55.391735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:55.392620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:55.398051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 302/595] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:55.399028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.404905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.405173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.405390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.405668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.406337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:55.406642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.407989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.425873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 294/596] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:55.426812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.483314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.542895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6294c-4e9d-4814-ba90-a1c33e2dfd5a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.543075 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2e0a4-391b-476c-b501-75705aff47e0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.579235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.579841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.580121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.580393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.581121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:55.581425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.583011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.589511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.590480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.591267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.591750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.592904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:55.594204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.597615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.610141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 303/597] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:55.610932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.620935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 295/598] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:55.620935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.669864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.725973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49fd17-f706-489d-b5d8-26514430f4ef None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.753305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4e237-d838-41a4-89bf-453e760f98c4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.766177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.770874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.771524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.771826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.772137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.782650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.784441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.796818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.797378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.797659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.797957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:55.798464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:55.799307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:55.800194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:55.802936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 304/599] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:55.804358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.812289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.814403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:55.834966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:55.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:55.837028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:55.837028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:55.837028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:55.837028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 296/600] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:55.837491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:55.924592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c3a92-a120-4f44-9c6f-8117a56d85cc None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.937458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:55.963444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:55.964726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:55.965234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:55.965780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/4637080dea3a4349b53a3097f4610465/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:55.978878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0fb9-267b-4cbd-a4a7-fcf890223c02 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:55.984812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=4637080dea3a4349b53a3097f4610465, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:55.986653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.016397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:56.017008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.017286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.017559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.018278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:56.018583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:56.020027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.033084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 297/601] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:56.033819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:56.058386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 305/602] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:55 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/4637080dea3a4349b53a3097f4610465/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:56.060349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:56.067241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:56.182895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ba677-0e58-4d70-9e4f-be959f641ba4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:56.215123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344eb2-ae06-4718-8e67-5c8c27031ff2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:56.232930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:56.234003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.234475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.234935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.235897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:56.236301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:56.237681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf46f9-e61c-4882-afec-684a9a109760 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.252720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 298/603] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:56.254223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a02ca-de1c-48f9-812a-2684d022aa9a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.254359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e052-30cc-40d6-810b-8d7ef0808189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:56.254611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a02ca-de1c-48f9-812a-2684d022aa9a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.254893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a02ca-de1c-48f9-812a-2684d022aa9a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.255094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e052-30cc-40d6-810b-8d7ef0808189 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.255391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e052-30cc-40d6-810b-8d7ef0808189 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.255677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e052-30cc-40d6-810b-8d7ef0808189 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/b9f9ea2ca628427bb165e1da9c0d1c7e/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.256453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 299/604] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:56.260330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a02ca-de1c-48f9-812a-2684d022aa9a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:56.283116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5e052-30cc-40d6-810b-8d7ef0808189 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=b9f9ea2ca628427bb165e1da9c0d1c7e, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:56.284659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5e052-30cc-40d6-810b-8d7ef0808189 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.361673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:56.381373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a02ca-de1c-48f9-812a-2684d022aa9a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:56.419527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 306/605] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:56 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/b9f9ea2ca628427bb165e1da9c0d1c7e/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:56.419790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:56.420712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.421067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.421415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.421836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.422310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.422977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.430038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:56.432014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.447448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:56.447653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:56.448526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:56.449406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 300/606] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:56.452628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:56.457678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:56.462233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:56.596903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 307/607] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:56.646019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b65c4-2080-4942-839d-6daedf18f76d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:56.688701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:56.689318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.689597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.689869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.690594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:56.690899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:56.692306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba61da-7e23-4847-9695-7c5f23d6e167 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.705943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 301/608] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:56.711397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:56.849923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039de13-922a-4bdf-a4d4-6164df78722d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:56.875546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf3ea-f23a-4734-9aaa-c362cda242a4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.876064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf3ea-f23a-4734-9aaa-c362cda242a4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.876141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf3ea-f23a-4734-9aaa-c362cda242a4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.877684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 302/609] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:56.883097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.883652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.884162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.890908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:56.908277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:56.908277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:56.911644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:56.912391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:56.912679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:56.913691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/2eeb9414ed49428684b9028c18476bba/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:56.934249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:56.942773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=2eeb9414ed49428684b9028c18476bba, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:56.946904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:56.985594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 303/610] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:43:57.002585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.023758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 308/611] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:56 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/2eeb9414ed49428684b9028c18476bba/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:57.135337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d1682-a9f4-4695-a2f8-f50f4cfc7824 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:57.172215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:57.183246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:57.183913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:57.184192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:57.184461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:57.197876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:57.207874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:57.208895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:57.210335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:57.211137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 304/612] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:57.216689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.304844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.351127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9506c73-7352-4f4d-916c-ddb3bb718903 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:57.390462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:57.391075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:57.391361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:57.391638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:57.392312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:57.392619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:57.394189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:57.404296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 309/613] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:57.409835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.472434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:57.528251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf92ff-7e2a-427c-8751-e1c5b01154e2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:57.547885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7ebc0-c4cb-41a2-9b44-e7eaa7b582ff None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:57.567332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:57.567977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:57.568261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:57.568541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:57.598331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:57.599865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:57.623029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:57.623730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:57.624059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:57.624339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:57.627445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:57.628350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:57.629260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:57.629260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:57.629260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:57.634025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 310/614] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:57.636980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.641536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:57.648186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:57.649115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:57.649967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:57.650600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 305/615] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:57.657030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.746304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:57.787389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468e4c5-b0ad-42d2-988e-8b3c0269b513 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:57.801916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eab556-fab4-4867-a6a7-48bc6eaacc7f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:57.818329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:57.831536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:57.832305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:57.832596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:57.832961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:57.833579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:57.833913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:57.835870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:57.849987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 311/616] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:57.855939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.903370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:57.904047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:57.904333 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:57.904629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:57.905356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:57.905659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:57.907116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:57.930848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 306/617] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:57.938912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:57.967516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbe944-fc5f-46c9-98e5-3eb3bfe1826e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.006362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.007824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.008120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.008401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.016748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.018755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:58.032824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:58.034042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:58.035369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 312/618] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:58.041136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.074282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:58.158168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d866d-5596-40b6-a17a-3d186fadd784 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.197427 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.198106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.198397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.198687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.199435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:58.199750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.201203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.223526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 313/619] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:58.235567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.249082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7682d13-2e1a-4b2c-b8ca-489e84a810dc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.264803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:58.287403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.288090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.288371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.289910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.306395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.307930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ce81db0b-7b69-4826-b05a-db0e061a79c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:58.324851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:58.326103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:58.327213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:58.327213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:58.327213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:58.327213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 307/620] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:58.327765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455d3f-45ec-42cd-8507-35860c14ba5f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.328049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455d3f-45ec-42cd-8507-35860c14ba5f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.328324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455d3f-45ec-42cd-8507-35860c14ba5f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.329940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 308/621] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:58.331625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455d3f-45ec-42cd-8507-35860c14ba5f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.376299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:58.401228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f828-c994-41a5-8c18-0ed3f7f14419 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.447460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455d3f-45ec-42cd-8507-35860c14ba5f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.448466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.449106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.449387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.449659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.463392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.465660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-92cc4ac2-a0aa-41aa-8975-bab185d679bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:58.484919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:58.486140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:58.487382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:58.487382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:58.487382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:58.487382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 314/622] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:58.487382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.487738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.487738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.487894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.489007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.489007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.489753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.490304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:58.492773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.493236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.509996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 309/623] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:58.510895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.513138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:58.514367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:58.545352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:58.607130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 315/624] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:58.608268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.655748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204376dc-cd42-48ce-be65-a4bcc04eda28 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.724557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.725761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.727119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.728325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.729267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:58.729972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.733128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5920ca-74be-435c-ac56-a01ac3d550d6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.733750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:58.747073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 310/625] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:58.748599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1ebc-ba17-4d67-8268-519d6d8eb74e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.749079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1ebc-ba17-4d67-8268-519d6d8eb74e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.749575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1ebc-ba17-4d67-8268-519d6d8eb74e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.751306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 311/626] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:58.752326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1ebc-ba17-4d67-8268-519d6d8eb74e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.783268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543772-f8f4-42b1-9f16-d0374ff26f94 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.821331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.821957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.822291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.822710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.831732 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.833365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a9b3e10a-9a3f-4f0d-825e-8185178053e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:58.846991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:58.847981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:58.848944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:58.853123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 316/627] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:58.853687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:58.855673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.856135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.856561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.870750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1ebc-ba17-4d67-8268-519d6d8eb74e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:58.877366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:58.878885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:58.907193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:58.907802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:58.908087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:58.908361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:58.917126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:58.918573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:58.932840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:58.934009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 312/628] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:58.935196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:58.965354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 317/629] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:43:58.966434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.034582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.064493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae7b95-f093-4d38-ad7f-2b1aa8724563 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.082477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6242f-0f2b-4ae0-952c-756cd15c8495 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.103587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.104549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.104976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.105410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.106408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:59.106914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.108774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.121476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.122317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.122706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.123086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.130734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 313/630] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:59.132755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.136822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:59.137447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:59.142944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:59.148036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:59.148911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:59.150083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 318/631] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:59.151206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.262022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0c9c-36b6-416a-aac5-ea700562aaed None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.268653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2520ddf-5cab-4fbf-97ce-2db4bd176642 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.286036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.296321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.296949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.297232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.297510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.298256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:59.298565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.300015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.308386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.309076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.309480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.309761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.320310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 314/632] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:59.320310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.384328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.385835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.389009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.429960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029625-b5ce-47d5-915f-44f08f861261 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.467859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.468465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.468746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.469021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.469678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:59.470016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.471368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.480838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 315/633] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:59.481733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.502990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 319/634] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:43:59.504207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.520198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.586860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d901d7-f5ac-4856-a3fb-fda651fc4d3d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.604941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8603-c408-4ac7-95df-d7e5997665f6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.614740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.624152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130863c-7e2f-415c-8714-3d9f7a415cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.624781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130863c-7e2f-415c-8714-3d9f7a415cbb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.625116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130863c-7e2f-415c-8714-3d9f7a415cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.625341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130863c-7e2f-415c-8714-3d9f7a415cbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.636286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130863c-7e2f-415c-8714-3d9f7a415cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:59.639325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.639976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.640252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.640524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/f0506db33e0e47aaa47a9a3fd9bc2f98/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3130863c-7e2f-415c-8714-3d9f7a415cbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:59.641053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:59.642192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:43:59.643222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:59.650485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 316/635] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:59.650485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4a5fe-ae52-46ef-ad65-f0722a1dea0d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.650485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4a5fe-ae52-46ef-ad65-f0722a1dea0d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.650485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4a5fe-ae52-46ef-ad65-f0722a1dea0d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.650485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 317/636] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:59.654395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4a5fe-ae52-46ef-ad65-f0722a1dea0d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.657679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=f0506db33e0e47aaa47a9a3fd9bc2f98, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.659166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.726316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 320/637] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:59 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/f0506db33e0e47aaa47a9a3fd9bc2f98/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:59.727704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.762648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4a5fe-ae52-46ef-ad65-f0722a1dea0d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.804377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.805371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.805656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.805953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.818141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.819922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.821890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:59.837463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:59.838829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:59.839730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:59.839730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:59.839730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:43:59.839730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 318/638] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:59.839952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:59.869214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56790643-044f-499a-9142-a64de86d5e67 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.886771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.913808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:59.914404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.914681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.914955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.925783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:59.927278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-fda41bf1-1442-4e9a-99fd-692998717b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:59.943314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:59.944630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:59.945539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:59.945539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:59.945539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:43:59.945539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 321/639] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:59.946742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:59.946742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:59.946742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:59.962566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e0ae-c5e5-454e-bf71-720f56f56e3f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:59.965674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:59.966888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:00.007040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.007651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.007932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.008206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.009390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.009710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.011129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.027573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 319/640] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:43:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.027573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.039023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 322/641] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:00.040748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.107620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.130317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.138285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba9228-8641-4669-81aa-a3efa3a512ee None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.168983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8270-80fb-4836-9e0a-258b8c56aa1d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.184124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.185232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.186113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.186906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.187771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.188436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.190410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.203842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 320/642] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.205049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.209554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.209554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.209837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.210127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.210778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.211086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.212528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.226308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 323/643] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.227119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.310843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0da1dd-55d4-4d42-ab56-49892f7edb08 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.339284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c329d5-7fa9-4347-a13a-fd093916c161 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.346978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.347819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.348555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.348840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.349112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.362233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.363866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.375806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.376438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.376715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.376983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:00.379703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:00.380892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:00.382730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:00.382730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:00.382730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:00.386165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 321/644] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:00.386942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.392216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.393627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.403971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:00.408314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:00.409440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 324/645] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:00.418631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.508954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fdb9b-e6ec-4a61-8e61-056408e2e92b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.514847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ee33-9728-4ff4-9583-3a9dd95e4b95 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.544994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.545712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.546050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.546328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.550704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.551329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.551609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.551882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.553060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.555690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.562757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:00.567669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:00.568523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:00.569706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:00.570286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:00.571106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 325/646] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:00.571924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.578390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 322/647] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:00.579177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.604715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.666146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.683428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb368-fe82-4f61-8be0-c331c3094df7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.697172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b2289-270b-4b8c-942e-80ba501ecca6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.721289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.722095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.722380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.722654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.723302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.723605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.725001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.734455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.735071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.735352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.735628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.736352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.736656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.743369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 326/648] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.743369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.743712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.768237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 323/649] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.778964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.838129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.853566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8e20-e485-4257-8f02-4a7610f47572 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.886109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a4a31-fefe-4b19-9e7d-d3ac15151ae5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:00.893284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.893917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.894203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.894539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.895233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.895539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.897553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.915003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 327/650] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.915003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:00.922513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:00.928821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:00.929621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:00.929919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:00.930309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:00.930998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:00.931294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:00.932645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:00.941936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 324/651] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:00.941936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.024932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8b90-fe1e-4540-9001-42c78cd3bfe2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.045286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd61952-4996-4e36-8805-c6fa149179a0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.062412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.063696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.064153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.064680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.074566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.076268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.081491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.081491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.081838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.082115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:01.088000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:01.089009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:01.090367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 328/652] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:01.091824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.100651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.157116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.166831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.168272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.204447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3bb75-c890-4101-b38c-84dfade26da9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.242030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.242649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.242937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.243215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.243949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=cf5c9c7caaf4451a9b1ca1a1095408d1) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:01.244263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.245963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.252312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 329/653] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/role_assignments?user.id=cf5c9c7caaf4451a9b1ca1a1095408d1&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:01.253129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.290087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 325/654] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:01.291882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.349777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.397757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57e465-3cd6-4007-820c-d48d29d82c37 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.408917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.421496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ecd0d-f457-46af-99fb-d9405de355d9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.441199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.442243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.442764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.443250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.460572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.462775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.477787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.478433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.478711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.478979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.479711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:01.480013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:01.480337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:01.481235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:01.482189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:01.482189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:01.482189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:01.482361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.482444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 330/655] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:01.482924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.512236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 326/656] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:01.513109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.613857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97491edc-4e78-4e35-a6ff-ba57d7bb8418 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.632813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea7102-7729-423f-b2d9-054cae11b0f1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.663883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.665243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.666109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.666402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.673538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.674475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.675444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.675944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.677235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:01.677754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.678545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.679775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.682877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.684565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.695250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 327/657] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:01.696649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:01.701505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:01.702711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:01.703811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:01.703811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:01.703811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:01.705151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 331/658] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:01.706159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.715534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.810283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ed725-4e4e-4e7b-aba3-979a2045e7c7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.817770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd43066-324f-480b-9f17-32c61e230d43 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:01.848526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874179d-ce93-4965-96b6-0fd06eda21f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.849466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874179d-ce93-4965-96b6-0fd06eda21f8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.849830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874179d-ce93-4965-96b6-0fd06eda21f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.850163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874179d-ce93-4965-96b6-0fd06eda21f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.856747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:01.857378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:01.857656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:01.858052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:01.866482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.867975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:01.879844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:01.881013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 332/659] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:01.883286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:01.921012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.933126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874179d-ce93-4965-96b6-0fd06eda21f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:01.934607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874179d-ce93-4965-96b6-0fd06eda21f8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:01.946660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:01.992691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd41bc-fe06-45fb-a947-18ab40e041d8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.030174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.030794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.031188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.031470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.032115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:02.032420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.033938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.043277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 328/660] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:02.043277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2574b4-ac8c-4214-a2e9-914cf03629a0 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.043277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2574b4-ac8c-4214-a2e9-914cf03629a0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.043277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2574b4-ac8c-4214-a2e9-914cf03629a0 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.043277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 329/661] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:02.046019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2574b4-ac8c-4214-a2e9-914cf03629a0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.047384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 333/662] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:02.048220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.152686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.159555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7454-f14b-4128-9dd2-1e798c3b7b19 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.169521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2574b4-ac8c-4214-a2e9-914cf03629a0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.181878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.195545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.196188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.196469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.196735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.207306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.208857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.210333 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.211057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.211518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.211772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.212502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:02.212811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.214232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-1b6c42cf-3897-4c54-8b86-f53154bf0a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:02.227336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:02.228600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:02.229690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:02.229690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:02.229690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:02.233279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 334/663] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:02.234456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.234739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.235014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.241286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 330/664] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:02.241286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.254348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:02.255459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:02.327586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 335/665] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:02.328453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.355056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302cc01-5ce1-4828-bd53-b7f6e01302e9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.394322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.395379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.395848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.396310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.405438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.407332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:02.420421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:02.421471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:02.422389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:02.429708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 331/666] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:02.430913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.433616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.460581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295abede-c4b4-4983-8ad9-b381b386fecd None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.465350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.509351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.510388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.510876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.511660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.513287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:02.513957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.515949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.537138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 336/667] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:02.538622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.555177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d931e-a17a-457e-9ecc-085996f4c45a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.595476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.599160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.599549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.600116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.674186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.675757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.678964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9c509-9151-436a-81a9-febc23dd615d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.703816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.715771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.716388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.716671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.716952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.717651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:02.717993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.719449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.722744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.733690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 337/668] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:02.734723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.792042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 332/669] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:02.793280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.850389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855a30-c052-4aa5-8a33-e1578ea5192d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.887456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.888094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.888380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.888658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.895705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:02.897283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:02.906721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570dcbff-57ae-4fdb-9625-8f05d94de50e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:02.908985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:02.910063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:02.911164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:02.915031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 338/670] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:02.916172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.945362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:02.952503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:02.953295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:02.953618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:02.953919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:02.965566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:02.970639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:02.971598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:02.972449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:02.974620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 333/671] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:02.974620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:02.986558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:03.041242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f30b-e0f2-408a-83ed-10921fc52f02 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.079039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.080043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.080564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.081094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=4637080dea3a4349b53a3097f4610465) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:03.081407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.082871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.087170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57a806-edd4-458a-a68d-fe0dbfce6ed3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.089933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 339/672] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/role_assignments?user.id=4637080dea3a4349b53a3097f4610465&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:03.090763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.148063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.149795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.150647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.151166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.168506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.170709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:03.192353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:03.193394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:03.195094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:03.195094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:03.195094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:03.197974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 334/673] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:03.199165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.222252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ffe81-a4a5-46ba-a951-567ac052cff4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.246935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:03.263062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:03.264338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.265135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.265569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.266025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.267684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:03.268151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.269983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.287698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 340/674] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:03.288757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.338496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd0231-5bdc-4a44-9de6-1e466d64aa2b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.400769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.402257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.402976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.403695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.405038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:03.405818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.408672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.420686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 335/675] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:03.421935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.452780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac2157-78a9-48d9-a91c-e7b27b15d101 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.506144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.507569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.507918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.508205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.508931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:03.509246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.511844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c15b11-91cc-4b86-acaf-8a87dd01dfdb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.530057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 341/676] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:03.532397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd650830-9c04-4c50-8e12-1a7f9502e865 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.533155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd650830-9c04-4c50-8e12-1a7f9502e865 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.534655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd650830-9c04-4c50-8e12-1a7f9502e865 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.536716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 342/677] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:03.538957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd650830-9c04-4c50-8e12-1a7f9502e865 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.571625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968606e2-0b26-433e-8cdb-0a29f1453eeb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.581030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:03.592454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:03.633070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.633793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.634246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.634620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.635438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=b9f9ea2ca628427bb165e1da9c0d1c7e) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:03.635821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.637351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.644987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 336/678] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/role_assignments?user.id=b9f9ea2ca628427bb165e1da9c0d1c7e&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:03.646092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.730655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd650830-9c04-4c50-8e12-1a7f9502e865 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.796266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.797068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.797421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.797779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.814845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.817511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.821170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e4a9-a6ff-44b7-b4e6-2e76d98e53d9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-2f146b01-af6a-4278-8b5d-9e0ce45748df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:03.840585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:03.841605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:03.842565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:03.842565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:03.842565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:03.842565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 343/679] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:03.843326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.843497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.844567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.864961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:03.866936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:03.873156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:03.873793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:03.874158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:03.874436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:03.881639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:03.883101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:03.895467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:03.896397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 337/680] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:03.898171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:03.932721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:03.963278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 344/681] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:03.969667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.015348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:04.055992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6db82-68c2-4d66-9b5f-a027ae6d7707 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.095439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.096082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.096322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.096506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.097147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:04.097445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:04.098948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:04.111869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 338/682] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:04.117676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.198007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859d013-bb90-4409-91d5-18330286ef75 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.234317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:04.260800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c9146-ff15-46d6-a993-5e3ce6d8a5c4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.276048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.277369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.277801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.278258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.279128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:04.279589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:04.281187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:04.305269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 345/683] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:04.306178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.321249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.321969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.322256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.322587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.330425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:04.331866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:04.345523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:04.346536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:04.347481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 339/684] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:04.348085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.416830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:04.437031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a8f71-563d-4874-aae5-1cbf486d93d5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.469801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d8fa3-2fa0-452b-8525-7e087373cde2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.474506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.475205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.475682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.475965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.487330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:04.493786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:04.495062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:04.495925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 346/685] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:04.496616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.522468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.523498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.524025 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.524510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.525442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=2eeb9414ed49428684b9028c18476bba) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:04.526087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:04.527898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:04.538261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 340/686] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/role_assignments?user.id=2eeb9414ed49428684b9028c18476bba&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:04.539522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.565932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:04.614610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ee23-11f5-4e3b-9a58-a119d0cd0d1d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.659344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.659986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.660270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.660549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.661308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:04.661619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:04.663840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:04.699465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:04.705482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 347/687] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:04.712515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.755227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca2452-2086-45ce-a28d-5f3e4b297507 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.800249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.800879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.801166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.801446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.802441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:04.802791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:04.806283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:04.816346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 341/688] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:04.822906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:04.859461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2544fd-9546-43da-8249-b1f406571c6d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.902128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:04.902671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:04.902949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:04.903223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:04.920947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:04.964508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ff544-1e5e-4107-9f99-69a60a4c48d3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:04.996166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:05.000086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.002365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.015579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.016582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.016879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.017438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.034973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.036434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:05.067429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:05.069283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:05.070736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:05.070736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:05.070736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:05.070736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 342/689] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:05.077470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.174913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 348/690] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:05.183126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.220356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dde33c-7feb-46be-a93e-2bb8c3f00e40 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.272225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:05.283622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.284352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.284695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.285050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.285766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:05.286177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.287728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadfff8-c6a4-432c-bad5-c77686c4e626 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.301324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 343/691] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:05.303342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dc9e2-652a-484c-925c-9c94ca20e11e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.303635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dc9e2-652a-484c-925c-9c94ca20e11e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.303913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dc9e2-652a-484c-925c-9c94ca20e11e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.309249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 344/692] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:05.310172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dc9e2-652a-484c-925c-9c94ca20e11e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.340433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:05.343750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d12241-b23d-4063-8513-5bb28491d1ed None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.386459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.387309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.388137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.388137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.400036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.401723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-4e5ff90b-6a50-456c-8633-5e976dd3aedb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:05.419658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:05.420873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:05.421989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:05.421989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:05.421989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:05.427780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 349/693] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:05.429689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.430092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.430468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.449043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dc9e2-652a-484c-925c-9c94ca20e11e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.452326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:05.453425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:05.494829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.495469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.495749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.496022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.507019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.509013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:05.525550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:05.526665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:05.527797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:05.530522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 345/694] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:05.531606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.538483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 350/695] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:05.539510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.619361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:05.692695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:05.712637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547df88-671b-4798-a1a2-761c543d8e76 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.713301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d624-b7ac-440e-82bd-9bee25231409 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.753093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.753707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.754058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.754335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.755140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:05.755500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.758649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.777124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 351/696] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:05.782173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.782705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.783290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.783779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.784256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.785106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:05.785641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:05.787750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:05.814822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 346/697] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:05.821628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:05.899017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2507ed-d344-4be9-bb9a-451209a05561 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.936408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef9a37-3599-49fc-955b-6a7b45fdefe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:05.937070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef9a37-3599-49fc-955b-6a7b45fdefe5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:05.937357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef9a37-3599-49fc-955b-6a7b45fdefe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:05.937632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef9a37-3599-49fc-955b-6a7b45fdefe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:05.949963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:05.975612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02e8f-12cd-4f95-9fc5-5f9d0602b836 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:05.978492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.014283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef9a37-3599-49fc-955b-6a7b45fdefe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.015757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef9a37-3599-49fc-955b-6a7b45fdefe5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.018357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.019001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.019291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.019572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.026606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.028737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:06.040738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:06.041666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 347/698] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:06.043011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.127809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 352/699] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:06.130067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e8f5-81b4-47bb-a930-78d2e333e587 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.130809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e8f5-81b4-47bb-a930-78d2e333e587 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.131886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e8f5-81b4-47bb-a930-78d2e333e587 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.133651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 353/700] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:06.134576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e8f5-81b4-47bb-a930-78d2e333e587 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.156772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973cf46-ab56-4799-8d21-5d664537a80f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.181616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.199646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.200334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.200566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.200845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.213781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.215281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:06.232649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:06.234389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:06.235287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:06.235287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:06.235287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:06.235287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 348/701] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:06.235287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.255732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.257122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e8f5-81b4-47bb-a930-78d2e333e587 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.295132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.295744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.296025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.296299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.297227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:06.297336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.298795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adf146-2459-4fba-9ef9-6e7108349a30 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.315137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 354/702] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:06.316268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.316560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.316836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.336525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:06.337713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:06.357397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f3b4-bdcd-40dd-9576-cd1983daaa97 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.395491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.396233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.396574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.397068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.407992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.409424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.418047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 355/703] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:06.418047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:06.425765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:06.428782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:06.429719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:06.429719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:06.429719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:06.429719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 349/704] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:06.429719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.464842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.532044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.559730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6731a9-877a-4da9-9b8a-353ceea26c60 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.569716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ed18f-3dbb-49dd-896b-adec301e4f2d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.600175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.600797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.601075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.601947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.602148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:06.602377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.603875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.622352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 356/705] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:06.623267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.626489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.627479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.627975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.628532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.629470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=f0506db33e0e47aaa47a9a3fd9bc2f98) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:06.630006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.632344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.639623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 350/706] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/role_assignments?user.id=f0506db33e0e47aaa47a9a3fd9bc2f98&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:06.640705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.788868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.812491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077cdba-208e-43c1-8978-0866b3b1f1be None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.820465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42272ae-7283-4c12-87d0-f7ea6c4f7105 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:06.853746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.854461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.854674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.854955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.855971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:06.855971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.857753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.862094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:06.869210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f53f6e-7b8f-4410-9cf9-34c73c55c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:06.872993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f53f6e-7b8f-4410-9cf9-34c73c55c9d7 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.872993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f53f6e-7b8f-4410-9cf9-34c73c55c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.872993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f53f6e-7b8f-4410-9cf9-34c73c55c9d7 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.872993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f53f6e-7b8f-4410-9cf9-34c73c55c9d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:06.873410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 357/707] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:06.873489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f53f6e-7b8f-4410-9cf9-34c73c55c9d7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:06.873753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:06.965823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 351/708] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:44:06 2022] POST /identity/v3/users => generated 254 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:06.967413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733c583-e979-46be-9d54-e405d8bf2c83 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:06.967703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733c583-e979-46be-9d54-e405d8bf2c83 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:06.968016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733c583-e979-46be-9d54-e405d8bf2c83 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:06.973528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 352/709] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:06.973528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733c583-e979-46be-9d54-e405d8bf2c83 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.006978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b76c36-6968-4c7e-8f5f-cfb6f4a21e7f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.055818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.057021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.057955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.058710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.059596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.062111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-e4721b2e-fe20-4ec1-b189-c03556846758 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:07.077857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:07.079037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:07.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:07.089294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 358/710] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:07.089294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.089294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.089294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.113055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:07.115472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:07.118532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733c583-e979-46be-9d54-e405d8bf2c83 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.128682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:07.169330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.170917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.170917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.170917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.180639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.182670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:07.201515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:07.202508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:07.203385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:07.205990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 353/711] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:07.207229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.224523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 359/712] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:07.224523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.272950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:07.342412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d469b-2d84-4a56-97f2-1d852ba93217 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.370189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8986-8200-4081-b88d-b880cfa8dcea None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.378364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.378978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.379258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.379531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.380161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:07.380461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.382051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.387037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 354/713] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 13:44:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:07.399092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.416966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.417593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.417872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.418170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.418873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:07.419185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.420621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.438011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 360/714] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:07.439768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.444203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:07.537159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485310cc-e894-4c16-8452-653a319f1886 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.566253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:07.590854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.591580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.591865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.592135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.592827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:07.593079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.594839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.602131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 355/715] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 13:44:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:07.603531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.643088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37244-d703-4c18-bf9d-3398e2ff9dd0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.698045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.701951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.701951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.701951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.701951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=cf5c9c7caaf4451a9b1ca1a1095408d1) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:07.701951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.702953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.715961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 361/716] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:07 2022] GET /identity/v3/role_assignments?user.id=cf5c9c7caaf4451a9b1ca1a1095408d1&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:07.775889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee65ed7-12de-4fc9-9ea6-f0ecc7168844 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:07.818872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:07.819884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:07.820315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:07.820735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] PATH_INFO: `/v3/services/network` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:07.821559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:07.823110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:07.836099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:07.837351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:07.838396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:07.840999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 356/717] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:07 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:07.846850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:07.928839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:07.965699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644e1f3-c00b-4b74-bdf7-cc8b31808e50 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:08.011371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:08.012035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:08.012444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:08.012727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:08.013632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] RBAC: Adding query filter params (name=network) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:08.013972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:08.015447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44786cd-be94-4782-9525-00a1c6066b9f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:08.025948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 362/718] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:08.031895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:08.106730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:08.183455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527637a6-36c7-46bd-88ed-b777eadb826e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:08.230643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:08.231279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:08.231568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:08.231847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:08.232494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:08.232800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:08.234196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb091e6-cd00-4927-829b-0aa5f80a88e4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:08.239489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 357/719] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:09.054670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd0b4-b73f-4f90-adae-9c07d1ac0951 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:09.055200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd0b4-b73f-4f90-adae-9c07d1ac0951 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:09.055200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd0b4-b73f-4f90-adae-9c07d1ac0951 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:09.056856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 363/720] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:09.062229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e0eb-f3f4-49a9-9f21-e6cbd71800de None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:09.062657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e0eb-f3f4-49a9-9f21-e6cbd71800de None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:09.062830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e0eb-f3f4-49a9-9f21-e6cbd71800de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:09.090771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-40f0e0eb-f3f4-49a9-9f21-e6cbd71800de None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:09.091982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-40f0e0eb-f3f4-49a9-9f21-e6cbd71800de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:09.182010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 358/721] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:09.397822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c6a98-267e-4ac8-9ea5-99c2134d3b2d None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:09.398508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c6a98-267e-4ac8-9ea5-99c2134d3b2d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:09.399046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c6a98-267e-4ac8-9ea5-99c2134d3b2d None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:09.400594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 364/722] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:09.406291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:09.406834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:09.407310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:09.435407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:09.436954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:09.472223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:09.589932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c6a98-267e-4ac8-9ea5-99c2134d3b2d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:09.670126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 359/723] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:09.734266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c6a98-267e-4ac8-9ea5-99c2134d3b2d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:09.766133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:09.776414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:09.777053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:09.777334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:09.777604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:09.798796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:09.803810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:09.804753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:09.805586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 365/724] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:09.810760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:09.923327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d35f85-6d97-4ac8-9f6b-589eae18b665 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:09.958546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:09.959398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:09.959761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:09.960042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:09.960712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:09.961021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:09.962421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965786f-c1e7-4a49-8dab-c5f4a3a5bce5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:09.971447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 360/725] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:09.979853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.014692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa03a0d-2b44-4841-8f8e-313d21c58e23 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.015037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa03a0d-2b44-4841-8f8e-313d21c58e23 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.015323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa03a0d-2b44-4841-8f8e-313d21c58e23 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.016785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 361/726] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:10.023337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa03a0d-2b44-4841-8f8e-313d21c58e23 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.055459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.120044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de1dc-c763-4fa6-99df-0cd0c1c0a1e6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.139180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa03a0d-2b44-4841-8f8e-313d21c58e23 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.164095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.164976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.165403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.165856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.178884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:10.180737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:10.183633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8f6f-a665-4754-a98b-bf6222b98e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.184272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8f6f-a665-4754-a98b-bf6222b98e6f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.184551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8f6f-a665-4754-a98b-bf6222b98e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.184825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8f6f-a665-4754-a98b-bf6222b98e6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-fdcca603-e0e9-4555-8306-ca6b44de69bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:10.200217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:10.201345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:10.202232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:10.202232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:10.202232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:10.202232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 366/727] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:10.202960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.203118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.203394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.207117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc8f6f-a665-4754-a98b-bf6222b98e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a8bc8f6f-a665-4754-a98b-bf6222b98e6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:10.212889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:10.213776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 362/728] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:10.219463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab54d3-5b25-496e-8fbd-ace141113a35 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.220190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab54d3-5b25-496e-8fbd-ace141113a35 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.220190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab54d3-5b25-496e-8fbd-ace141113a35 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.220190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 363/729] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:10.220190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab54d3-5b25-496e-8fbd-ace141113a35 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.227928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:10.229141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:10.273572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.304633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 367/730] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:10.305569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.324577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.342772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab54d3-5b25-496e-8fbd-ace141113a35 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.391237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.392152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.392435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.392776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.393698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:10.394497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:10.396195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814b723-80ad-4aaa-a548-c82d43d85ec6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:10.412989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 364/731] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:10.414303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.414590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.414864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.438902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:10.440572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:10.459756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23061-0093-4108-b22d-5b9acc28dddb None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.503790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.504751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.505140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.506290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.507038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:10.507412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:10.509235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:10.521276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 368/732] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:10.522728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.531483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 365/733] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:10.532447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.606256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.618270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.641988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd7eda-0c13-4812-9e00-9b0d18e208d1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.687369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.688009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.688288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.688559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.702743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:10.704466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:10.709954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc7637-80e0-4cd6-9e07-f43d0c9f10d6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:10.727007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:10.728245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:10.729397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:10.729397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:10.729397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:10.729397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 366/734] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:10.729397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.760982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.762064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.762444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.762811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.774107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:10.776615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:10.792543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:10.793539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:10.794522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 369/735] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:10.796004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.859990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e33fe5-de3c-4966-a451-c8a2e11d64bc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.890614 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.898421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.899058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.899339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.899730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.910665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:10.913108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:10.916289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:10.917282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:10.918140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 367/736] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:10.919300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:10.919371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcce554-488c-46a4-b5c6-9b57cdde9710 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:10.952672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:10.953329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:10.953550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:10.953818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:10.954612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:10.955245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:10.957009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:10.969389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 370/737] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:10.970196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.020350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c885-f169-45c3-88c0-a191521030bd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.054583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.055195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.055475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.055753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.056495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:11.056803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.058253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001be41d-270c-4777-a9a5-1f7f731d4d62 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.074191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bfbb9-1486-433f-b9bd-25981dc4a5b2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.075177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 368/738] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 13:44:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:11.076391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334189-c423-4689-b4ca-642afc198f50 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.076673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334189-c423-4689-b4ca-642afc198f50 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.076949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334189-c423-4689-b4ca-642afc198f50 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.078312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 369/739] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:11.078846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334189-c423-4689-b4ca-642afc198f50 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.115531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.116141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.116416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.116686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.117335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.118731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.128603 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.182243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334189-c423-4689-b4ca-642afc198f50 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.215451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.215965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.216243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.216552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.216803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 371/740] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:10 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:11.217271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:11.217506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.217704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.218887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.229379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 370/741] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:11.231210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.243280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.348301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1526be7-2a3f-4f46-9163-5c03be9da3c7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.361751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aef5b-dbee-4e69-a6ac-f60ec9b08464 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.374299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.388514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.389207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.389494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.389770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.400627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.402093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.404620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.405597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.406143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.406614 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-71ce33b2-5453-4798-a479-94f7ee18e134 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:11.415424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:11.416391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:11.417314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 372/742] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:11.418453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.418453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.418705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.437831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:11.439104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:11.486296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.487757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.515046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 373/743] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:44:11.515970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.518443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.609306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 371/744] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:11.610503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.622721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.635918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128a2ca-ae40-49d6-8093-528587a07609 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.671958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.672602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.672882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.673153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.683906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.685391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:11.701591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:11.702881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:11.703770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:11.703770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:11.703770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:11.703770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 374/745] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:11.704003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.725749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f3aa7-2089-4bd7-8163-72d467281818 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.763416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.763956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.764241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.764517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.765357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:11.765671 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.767251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.767822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.786456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 372/746] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:11.788108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.814013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9219-dc72-491d-a60c-d439a90ca826 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.853101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.853734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.854042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.854319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.855137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:11.866483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.867948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:11.884493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:11.885499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:11.886398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:11.886398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:11.886398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:11.886398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 375/747] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:11.886638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.897552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b74f28-29ab-41cc-8752-caf984970fa6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:11.935691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:11.936310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:11.936598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:11.936873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:11.937516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:11.937825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:11.939219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:11.956727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 373/748] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:11.956727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:11.999035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74090d39-63c9-46f5-aaa7-faba4614edd6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.007029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.037765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.039023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.039442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.040004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.063985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2050-c875-4a7b-8d8c-50bd8c404663 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.093831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.103371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.103974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.104250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.104526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.105172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.106552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.112518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.114028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.219072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 376/749] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:11 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:44:12.219912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.230238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.241937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 374/750] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:11 2022] POST /identity/v3/services => generated 240 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:12.241937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.351427 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a3863-b644-48ba-9b3b-cc59cd898448 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.393423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3db533-ba29-4e6a-97ba-dd98145b5efa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.399430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.400053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.400335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.400612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.401258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:12.401561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.403093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.415872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 377/751] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:12.416520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.435908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.436938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.437484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.438002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.446795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.448702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:12.463047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:12.464020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:12.464948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:12.467404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 375/752] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:12.468494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.486436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.533699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.535147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f7ff-721d-41a0-a941-ee1bf12777c0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.581556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.582406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.582783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.583159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.583277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016e10-e898-4886-8994-5f6bbb2a5c18 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.598507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.600230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:12.620021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:12.621211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:12.622208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:12.622208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:12.622208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:12.622208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 378/753] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:12.622208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.623871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.627982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.630008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.630008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.636889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.638599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:12.651637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:12.652784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:12.653936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:12.660183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 376/754] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:12.661177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.761696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac207ed-5a18-492e-805e-24ed8c9c0259 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.778411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46083df-ce2d-4519-9acb-feab0acf3324 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.780347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.783562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:12.804999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.805786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.806194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.806640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.807501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:12.807974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.809553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.828263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.829166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.829611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.830083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:12.830899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:12.831346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:12.832839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:12.836145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 379/755] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:12.837194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.849935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 377/756] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:12.851018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:12.958877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc048-040a-4ea2-a2b0-bf36b5d733cb None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.969074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f91b7-8fff-4f2c-a6cb-8d8e04056677 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:12.997798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:12.998719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:12.999159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:12.999584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.000466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:13.000985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.002655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.009644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.010329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.010615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.010894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.017270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 380/757] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:13.018371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.034379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.035893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:13.053447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:13.054488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:13.055378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:13.055378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:13.055378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:13.059809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 378/758] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:13.060989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.064225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:13.080259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:13.138290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e3f8-a20d-4851-8891-6452c0be60fc None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.175067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.175856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.176242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.176618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.184716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.185136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f63395-c4ac-4ff7-b5d8-c9287cc40378 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.186510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:13.199870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:13.201548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:13.203248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:13.208684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 381/759] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:13.208684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.236957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.237533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.237817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.238374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.238895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=f0506db33e0e47aaa47a9a3fd9bc2f98) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:13.239315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.241142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd80e50-abaa-484f-9c9e-fcc524c80fff None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.251243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 379/760] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/role_assignments?user.id=f0506db33e0e47aaa47a9a3fd9bc2f98&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:13.251243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c63b82-bd5b-4c57-8f65-5b52c5ea2e7f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.251243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c63b82-bd5b-4c57-8f65-5b52c5ea2e7f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.251243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c63b82-bd5b-4c57-8f65-5b52c5ea2e7f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.254471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 380/761] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:13.254982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c63b82-bd5b-4c57-8f65-5b52c5ea2e7f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.316013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:13.327726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b44a-226b-447e-be5f-27f18ec6a058 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.355865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:13.369660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.370453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.370805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.371158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.371894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:13.372271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.373756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ad0c0-280e-49c2-901c-0ec8ce1ec8ab None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.388174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 382/762] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:13.389660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.390042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.390413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.393026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c63b82-bd5b-4c57-8f65-5b52c5ea2e7f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.410405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:13.412717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:13.439813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.440684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.440972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.441273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.443635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:13.443635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.446020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.469935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 381/763] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:13.469935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.503481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 383/764] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:13.504806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.582131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e03d4b-73c9-4d7c-898d-2c1633730c48 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.608578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:13.637820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3abb7-c494-47e6-bf70-f8dae10f5911 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.644221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.645138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.645617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.646193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.653687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.655997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:13.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:13.670841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:13.671722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:13.674192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 382/765] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:13.675015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:13.679922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.763696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.764247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.764830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.764830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.765551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=a910227bf388410196f8b201db25ca1b) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:13.765865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.767331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.790376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c1a1d-7cc1-45dc-8505-d102aa9e6536 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:13.795678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 384/766] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/role_assignments?user.id=a910227bf388410196f8b201db25ca1b&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:13.827023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.843706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:13.844674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:13.845185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:13.845458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:13.846220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:13.846528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:13.848735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:13.863331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 383/767] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:13.869559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:13.946292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:14.009207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fb536-f755-44b2-936a-04dc47208a43 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:14.043038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a52b8-7860-4e3e-adbd-55bd2c43462d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:14.074381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:14.075068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.075351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.075629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/cf5c9c7caaf4451a9b1ca1a1095408d1/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.079887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=cf5c9c7caaf4451a9b1ca1a1095408d1, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:14.081328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:14.086584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:14.087281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.087572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.087847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.088493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:14.089862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:14.101881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:14.103603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:14.104452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 385/768] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 1 13:44:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:14.109277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:14.124232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:14.151043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 384/769] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:44:13 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/cf5c9c7caaf4451a9b1ca1a1095408d1/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:44:14.236575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14849822-f870-4b87-a780-8e2e0b7cf7fe None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:14.277332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:14.277976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.278261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.278536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.279172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:14.279474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:14.280857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:14.286251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 386/770] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 13:44:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:14.292156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:14.430575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d450069-460c-4dda-8427-9787c3242a07 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:14.452602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:14.474566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:14.479450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.479450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.479450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.479450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:14.479450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:14.479450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2f27-6241-4213-b7d5-5723cc754241 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:14.483287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 385/771] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 13:44:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:14.490303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:14.575506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e908dd9-0f51-4b97-a510-2fe51b21d609 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.575837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e908dd9-0f51-4b97-a510-2fe51b21d609 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.576060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e908dd9-0f51-4b97-a510-2fe51b21d609 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.581999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 386/772] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:14.587546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.587834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.588111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.601751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:14.629029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7535c9-6921-483a-8689-d7c987e6b122 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:14.633029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:14.634259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:14.709362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:14.709362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.709362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.709362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.709761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] RBAC: Adding query filter params (interface=public, service_id=da20d5fc76d746e783d639a2bb3f2d43, region_id=RegionOne) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:14.709761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:14.709761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:14.714690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 387/773] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:14 2022] GET /identity/v3/endpoints?service_id=da20d5fc76d746e783d639a2bb3f2d43&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:14.725122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:14.782001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 387/774] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:14.860585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e67985-b8e1-41d0-a70c-d0f56563d81e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:14.899211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:14.900199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:14.900659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:14.901147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:14.902027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:14.903570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:14.909503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 388/775] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:14 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:15.129138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:15.249089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e947bef-e06c-4a1c-9b49-6b0593f42806 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:15.265791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:15.289699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:15.290700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:15.291181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:15.291673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] PATH_INFO: `/v3/services/network` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:15.292457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:15.294495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4043b58b-92d6-4391-857a-af8764b3e716 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:15.305210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:15.306086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:15.313421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 388/776] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:15 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:15.315087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:15.326252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:15.434664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0524f-2b18-4b3d-a348-f09bbdbd33ac None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:15.477559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501f03e-1854-4bec-a5a4-cb7ba9517e2f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:15.477925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501f03e-1854-4bec-a5a4-cb7ba9517e2f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:15.478176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501f03e-1854-4bec-a5a4-cb7ba9517e2f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:15.479776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 389/777] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:15.480466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:15.481538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:15.481824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:15.482131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:15.482834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:15.483145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:15.484858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:15.484987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:15.485192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:15.485469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:15.496058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 389/778] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:15.502284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:15.507661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:15.508954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:15.626275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:15.658805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 390/779] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:15.684396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca72b7-6990-459e-8e3f-2d02df882fd8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:15.728419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:15.729045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:15.729330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:15.729607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:15.731916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] RBAC: Adding query filter params (type=network) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:15.731916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:15.732182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:15.739362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 390/780] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:15.746011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:15.863961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634eb67-f1df-49bc-9eab-f51e8719ff66 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:15.872903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:15.901059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:15.901676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:15.901995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:15.902277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:15.902992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] RBAC: Adding query filter params (interface=public, service_id=f29f5db46f224d2a9527b1249e535db2, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:15.903303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:15.904672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:15.913496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 391/781] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:15 2022] GET /identity/v3/endpoints?service_id=f29f5db46f224d2a9527b1249e535db2&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:15.919004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:15.977965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:16.100486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1916bb-9707-4767-968f-0407b7667cfe None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:16.153367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53890-c7ce-4a26-b53c-3dc1e21d92c5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:16.157696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:16.169499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:16.170139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.170418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.170691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.171301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:16.172668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:16.182572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 391/782] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:15 2022] GET /identity/v3/services => generated 772 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:16.224280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f4aa1-1732-4f0f-b879-03507ea25d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:16.224955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f4aa1-1732-4f0f-b879-03507ea25d88 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.225235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f4aa1-1732-4f0f-b879-03507ea25d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.225504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f4aa1-1732-4f0f-b879-03507ea25d88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.246937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f4aa1-1732-4f0f-b879-03507ea25d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-855f4aa1-1732-4f0f-b879-03507ea25d88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:16.252471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:16.253467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:16.255250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 392/783] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:16.259609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:16.380305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b662cb6-6a21-49ef-8c3b-79557a3bd4af None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.380817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b662cb6-6a21-49ef-8c3b-79557a3bd4af None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.380817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b662cb6-6a21-49ef-8c3b-79557a3bd4af None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.382365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 393/784] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:16.391323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62744645-b6d0-4e85-92d2-ff04191f1734 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.391624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62744645-b6d0-4e85-92d2-ff04191f1734 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.391941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62744645-b6d0-4e85-92d2-ff04191f1734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.407626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36fa2d-552a-48f3-b497-e8139ac1f2cf None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:16.427571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-62744645-b6d0-4e85-92d2-ff04191f1734 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:16.429549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-62744645-b6d0-4e85-92d2-ff04191f1734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:16.451675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:16.452289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.452566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.452837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.453507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:16.453810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:16.455425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:16.459467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:16.465672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 392/785] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:16.471105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:16.514294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 394/786] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:16.515764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ecb5b-c186-4836-ba51-7e0f9544e284 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.516052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ecb5b-c186-4836-ba51-7e0f9544e284 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.516326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ecb5b-c186-4836-ba51-7e0f9544e284 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.521921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 395/787] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:16.527441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.528084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.528559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.549246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:16.550922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:16.641338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 396/788] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:16.682101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:16.747559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91a353-4697-4b10-a0b2-d451282876e6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:16.799622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:16.800271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:16.800553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:16.800830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:16.816744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:16.819261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:16.837673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:16.838684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:16.839554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:16.839554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:16.839554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:16.840159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 393/789] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:16.841009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:16.846361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:16.995307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062be26e-9d36-4412-b0c0-adb47f697db2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:16.997294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.006520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.010287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44c4c-b988-4166-8b10-17ec5277ff37 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.034054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.035110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.035493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.035864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.038030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:17.038453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.039942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.060809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 394/790] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:17.060809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.062924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.064016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.065400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.065926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.078639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:17.085550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:17.086514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:17.087330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:17.090396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 397/791] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:44:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:17.091534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.195858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62187a0d-85b7-4f50-956b-8212edcbb44c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.219135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1314c-5d62-431a-a75b-80e57a745a3d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.235207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.235862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.236128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.236406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.248576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:17.254462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:17.255717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:17.256704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:17.260572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 395/792] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:17.261431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.266560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.267397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.267892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.268322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.278704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.281036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.294162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:17.295355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:17.296455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:17.297399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:17.301252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 398/793] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:17.302161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.315966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.390366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a690909-e8b8-4fb9-b2e0-a8d1eb6655a9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.415184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a706-8950-4424-beac-c7f0ce6caaaf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.431783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.432795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.433252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.433774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.434622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:17.435106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.436755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.451728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 396/794] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:17.453053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.456579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.457201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.458605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.458894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.459582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:17.459876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.461253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.477280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 399/795] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:17.477280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.566190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.571901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.582228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2c9a5-45b0-4b18-ad0e-9bacc8d3d760 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.622166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f791b44-dd63-41f1-a35d-a1782e2b1024 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.632065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.632683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.632969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.633227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.633974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:17.634638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.636378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.659478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 397/796] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:17.660509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.667147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.668144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.668663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.669163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.681736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.683526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:17.700109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:17.701229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:17.702161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:17.702161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:17.702161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:17.709498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 400/797] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:17.710746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.775723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a8f81-5467-49a3-9176-8961056d639d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.813660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.814604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.815063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.815498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.818530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8327-190b-48d5-b190-7544aa47970b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:17.826567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.828062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.838082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:17.845495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:17.846602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:17.847486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:17.847486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:17.847486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:17.847486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 398/798] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:17.848520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.849809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:17.863429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:17.864064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.864484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.864897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.878199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:17.879642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-35df685f-307e-43fb-a6f1-89d7817d3433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:17.894609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:17.896452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:17.897356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:17.897356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:17.897356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:17.897356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 401/799] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:17.897591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c047-8b56-4fb1-844e-4a9ed438c000 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:17.898499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c047-8b56-4fb1-844e-4a9ed438c000 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:17.898499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c047-8b56-4fb1-844e-4a9ed438c000 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:17.899355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 402/800] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:17.899946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3c047-8b56-4fb1-844e-4a9ed438c000 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:17.966707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c0e4c-3d74-483f-af0e-997472cf4e8d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.003746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.004399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.004684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.005563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.006268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:18.006575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.008062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.018197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3c047-8b56-4fb1-844e-4a9ed438c000 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.021489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 399/801] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:18.026225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.059561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.060237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.060521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.060791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.061437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:18.061740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.063136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58ca10-829d-462c-baa3-ff2e5f52d17d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.075692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 403/802] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:18.077016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.077296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.077567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.094772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.096017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:18.097232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:18.128457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.143346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024d350-080f-4136-89d1-ec793ad4847e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.173609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 404/803] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:18.174356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.191000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.192348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.192847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.193598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.194910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:18.195434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.197591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.216728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 400/804] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:18.218457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.287650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5ceea-f2ee-463b-b9c9-d3c2b6791653 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.340480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.342132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.342440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.342720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.345209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b30358-322a-459d-ac33-65fdac8ef4b7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.351491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.353015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:18.366490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:18.367496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:18.368425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:18.372633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 405/805] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:18.373597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.375959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.396220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.397212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.397701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.398802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.405017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.409047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.411610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:18.424147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:18.431077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 401/806] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:18.431994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.481250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec63ab-2c4e-4394-ac45-d244cf708e22 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.516433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.517092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.517370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.517854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.525227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.526693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.533364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b41269-fe7e-4641-827a-c00ab29f7b8d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7bd15eb4-1a0b-4ac5-8123-4a16fc32d804 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:18.539283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:18.540488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:18.541611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 406/807] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:18.542207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30a44-9ea0-4e75-92f3-1f8df10477af None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.542207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30a44-9ea0-4e75-92f3-1f8df10477af None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.542535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30a44-9ea0-4e75-92f3-1f8df10477af None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.543811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 407/808] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:18.545353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30a44-9ea0-4e75-92f3-1f8df10477af None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.573672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.574375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.574724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.575036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.575658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:18.575966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.577406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.598264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 402/809] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:18.599559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.637154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.641329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.652595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30a44-9ea0-4e75-92f3-1f8df10477af None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.687673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.688310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.688602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.688873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.689600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:18.689931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.691355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.701707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505a55f-184b-4689-ba96-5c1beafedaec None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.709812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 408/810] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:18.710773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.737282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.737808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.738120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.738395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.738981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.740910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-ca822d9d-62c7-4710-9d4e-b097e5c766b6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:18.750036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:18.751224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:18.752064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 403/811] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:18.752861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.752861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.753097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.771707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:18.772905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:18.814796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f57c09-3726-4601-8ed1-6281b5d52a3e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.840057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 404/812] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:18.840908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.859943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.860939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.861270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.861546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:18.862401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:18.862636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:18.866335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:18.874791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.879710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 409/813] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:18.880431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:18.885273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:18.954805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c253-9f09-46ca-b698-b2704efdc12a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.991267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b46f35-2070-4486-b7ab-6227f3b73ee1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:18.993291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:18.994225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:18.994492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:18.994769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.006179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.007634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:19.025784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:19.026927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:19.027836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:19.027836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:19.027836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:19.027836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 405/814] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:19.028526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.035046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.035692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.035981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.036291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.112884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.114415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.117195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.117495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.133946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcede2-c584-46b2-9071-65539888fcd1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.168996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.169599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.169971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.170261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.171153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:19.171492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.172849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.177453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 406/815] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:19.178261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.217624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 410/816] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:19.218525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.285920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c7e9e-0188-4f63-8b85-1ab90193e005 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.320162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.320708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90603b14-e6ae-4510-b733-f3fb75d205b4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.320846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.321088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.321366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/f0506db33e0e47aaa47a9a3fd9bc2f98/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.323059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.323321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=f0506db33e0e47aaa47a9a3fd9bc2f98, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.324755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.354974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.355581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.355865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.356143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.356782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:19.357090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.358480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.374953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 411/817] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:19.374953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.394866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 407/818] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:44:19 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/f0506db33e0e47aaa47a9a3fd9bc2f98/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:44:19.395683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.407526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.496102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11227-7c9c-462f-9de5-7ab7d6a6406a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.514957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a7018-8169-4433-9066-b2d58f42c48e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.533882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.534582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.534867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.535144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] PATH_INFO: `/v3/services/network` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.535741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.537496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:19.551100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:19.552699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:19.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 412/819] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:19.555189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.561139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.561139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.561408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.561633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.562403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:19.562716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.564116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.570023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 408/820] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:19.570771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.592318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.651269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.675842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2cea4-b90c-4207-b7c9-4defb4042c5d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.683760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1072-4367-4ee6-9fc9-6b3d0dd70a3a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.717368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.718367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.718883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.719375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.720133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.720781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.721069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.721347 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.729013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.730502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.733130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.734827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:19.743015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:19.744228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:19.745551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:19.749154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 409/821] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:19.750074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:19.751964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:19.752868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:19.759734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:19.759734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:19.759734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:19.759734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 413/822] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:19.759734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:19.841819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.863352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ec60-95e3-4407-ae3e-6b30ca05bd20 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.864980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c78697-34ea-4c0d-9be5-998e55663a00 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:19.896733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:19.902398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.902574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:19.902954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.903059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:19.903229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.903323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:19.903500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.903604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:19.904131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:19.904224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.904435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:19.905528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.905786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:19.910524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 410/823] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:19.911866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.015339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077bd9c-c284-43a1-a14d-159f804a0af9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.026182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 414/824] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:20.026182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.074352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.075033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.075371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.075695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.076407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:20.076721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.078226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.099493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 411/825] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:20.099493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.160153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:20.160997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387335b-2ac5-40df-92fd-c8cbfe8d80c8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.219010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.219371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.219666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.219954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.220610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:20.220919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.222338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.226019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:20.226317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75192b8-7730-4017-affa-c9e76615c2f4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.236795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 415/826] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:20.243286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.275407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.276004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.277152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.277152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.277312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:20.277530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.278950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.289922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 412/827] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:20.289922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.384771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17af530-3572-4947-8795-a70a301f7830 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.407473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebba8f-49fd-4d78-bd0b-bf9be69402c8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.430486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:20.433094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.434407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.435108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.435536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/a910227bf388410196f8b201db25ca1b/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.445781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.446443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.446721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.446993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.454386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.455854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=a910227bf388410196f8b201db25ca1b, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.456003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.457577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:20.469808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:20.475035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 413/828] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:20.475911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.528652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 416/829] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:44:20 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/a910227bf388410196f8b201db25ca1b/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:44:20.530393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.597260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:20.644649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cd10e-6f6f-4568-a906-1e226fd06ff4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.679495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10614ef9-e7e7-4a18-aa07-0fad7a5ca00d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.710872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c604e9e-e903-4069-b483-d0172209cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.711262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c604e9e-e903-4069-b483-d0172209cabe None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.711567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c604e9e-e903-4069-b483-d0172209cabe None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.711849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c604e9e-e903-4069-b483-d0172209cabe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.712525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c604e9e-e903-4069-b483-d0172209cabe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.713957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c604e9e-e903-4069-b483-d0172209cabe None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.725622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.726597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.727732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.728396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.729170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:20.729516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.731439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.744807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 417/830] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:20.750977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:20.818002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 414/831] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:20 2022] POST /identity/v3/endpoints => generated 337 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:20.892358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886cf80-2c59-4240-936b-4e4521751600 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:20.902522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:20.945348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:20.946006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:20.946355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:20.946637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:20.947382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=cf5c9c7caaf4451a9b1ca1a1095408d1) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:20.947692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:20.949267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde3964-3618-490c-9c00-1c1ed0436544 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:20.956536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 418/832] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:20 2022] GET /identity/v3/role_assignments?user.id=cf5c9c7caaf4451a9b1ca1a1095408d1&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:21.487829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51cd2-28aa-445b-ba77-94e5ff1a8de6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:21.488855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51cd2-28aa-445b-ba77-94e5ff1a8de6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:21.489990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51cd2-28aa-445b-ba77-94e5ff1a8de6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:21.491826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 415/833] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:21.497960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:21.498506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:21.498989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:21.519169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:21.521348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:21.597307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 419/834] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:21.882703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51cd2-28aa-445b-ba77-94e5ff1a8de6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:21.918263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:21.995304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51cd2-28aa-445b-ba77-94e5ff1a8de6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.031570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ec87b-5133-4de0-928c-6e729013b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.032268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ec87b-5133-4de0-928c-6e729013b977 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.032543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ec87b-5133-4de0-928c-6e729013b977 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.032850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ec87b-5133-4de0-928c-6e729013b977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.043657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ec87b-5133-4de0-928c-6e729013b977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-123ec87b-5133-4de0-928c-6e729013b977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:22.048453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:22.049386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:22.050248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 416/835] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:22.058134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.161787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:22.167778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5e609-c025-4a4e-975a-4743615e974b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.179055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c3cbc-7d27-4897-b7ac-6e619d203dbc None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.179412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c3cbc-7d27-4897-b7ac-6e619d203dbc None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.179624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c3cbc-7d27-4897-b7ac-6e619d203dbc None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.181232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 417/836] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:22.186973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.187284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.187537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.207770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:22.209543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:22.209729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.210382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.210668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.210993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.211966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:22.213198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:22.214669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:22.226191 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 420/837] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:22.231468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.309923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 418/838] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:22.357020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f264975-20c2-451f-99de-7ed8fd4e21bb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.395685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.396387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.396675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.396954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.408544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:22.410014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:22.425652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:22.427174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:22.428071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:22.428071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:22.428071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:22.428242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 421/839] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:22.434036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.462888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:22.547439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b1fe-c2dc-4018-8a14-9593e2410ebb None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.587095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.587722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.588003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.588280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.588928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:22.589235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:22.590696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:22.608172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 419/840] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:22.613437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.620812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.628856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:22.710048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:22.730228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0999ab-d73a-4cb2-a3a5-9710f1787aec None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.734836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b2dac-8b32-4a49-82a7-e6cd80080000 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.772689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.773323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.773608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.773708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.773933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.774355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.774634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.774905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.775498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:22.776866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:22.782836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:22.784510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-ed16e872-e9d7-4460-b73b-83843b50ea79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:22.785797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:22.786907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:22.787752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 420/841] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:22.788560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b17339-8bff-4d4e-8f70-1d73727fdaaf None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.788560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b17339-8bff-4d4e-8f70-1d73727fdaaf None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.789047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b17339-8bff-4d4e-8f70-1d73727fdaaf None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.790126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 421/842] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:22.794791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b17339-8bff-4d4e-8f70-1d73727fdaaf None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-85ff76c5-fe67-42af-a5bc-d95b3a69449e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:22.804310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:22.805321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:44:22.806286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:22.806829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 422/843] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:22.807764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.808054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.808332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.828409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:22.829609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:22.897235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:22.901566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 423/844] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:22.902459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:22.933502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b17339-8bff-4d4e-8f70-1d73727fdaaf None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:22.976204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:22.976829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:22.977113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:22.977388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:22.978196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:22.978502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:22.979983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:22.990542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 422/845] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:22.995037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.014659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.034239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1204a-2777-4bcd-acb2-a9e9864c6684 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.075379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.075999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.076303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.076587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.077324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:23.077635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.079096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12034a52-09c4-45a0-8e9a-43ae926dc10b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.100297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 424/846] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:23.101388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959affee-0d28-4ac2-a5c0-3acf70ffd82d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.101678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959affee-0d28-4ac2-a5c0-3acf70ffd82d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.101982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959affee-0d28-4ac2-a5c0-3acf70ffd82d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.103352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 425/847] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:23.107233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959affee-0d28-4ac2-a5c0-3acf70ffd82d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.124294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33b5d6-39e6-4416-8551-d8f2ff9a0ca5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.161671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.162315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.162702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.162905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.163495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:23.163902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.165317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a034c2-130b-415b-b11c-c96b458ad66a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.177926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 423/848] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:23.177926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.177926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.177926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.193053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.200292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:23.216853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:23.246134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959affee-0d28-4ac2-a5c0-3acf70ffd82d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.283473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 424/849] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:23.284834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.286849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.287636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.287987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.288331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.291440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.371255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.373094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.416802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b30f-dd8c-4e9a-83e5-182d9d6345f1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.437167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.460422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.461084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.461395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.461656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.472580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:23.479411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:23.480368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:44:23.481191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:23.485649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 425/850] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:23.486724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.489249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 426/851] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:23.499847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.568984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.611307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c84e-d10e-479e-80ba-b2fba0b3be4c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.619246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbaf04-e2f0-4363-9301-ddddf4910f28 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.649828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.650525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.650809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.651080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.651741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:23.652044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.653416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.662893 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 426/852] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:23.663420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.668001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.668652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.668935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.669213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.680818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.682284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:23.697934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:23.698999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:23.699927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:23.699927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:23.699927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:23.699927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 427/853] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:23.699927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.714690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.781734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942231d2-f317-4e78-853f-349dc32523c0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.808842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.811474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc91c3-7d5e-49be-b2fc-e4e2f6bb279e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.819173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.819898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.820392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.820662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.821249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.822651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:23.832316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:23.833419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:23.834350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 427/854] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:23.835623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.854124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.855108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.855579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.855863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.867343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.869038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:23.885049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:23.886182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:23.887045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:23.887045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:23.887045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:23.887045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 428/855] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:23.887771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:23.948867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:23.953154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341b042-7755-43b1-a622-11322ddf6bb7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.988013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce9c9e-d373-4fce-a780-5a847fa7b1b7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:23.989943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:23.990776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:23.990958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:23.991192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:23.991837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:23.992143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:23.993814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.011355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 428/856] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.012272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.026658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.027323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.027602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.027873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.028526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:24.028832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.030231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.035015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 429/857] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.035824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.075646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:24.131506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed617cb-4867-40a6-ab28-92cba9d76b48 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.146742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4375e0d-1ee6-4913-afd3-4ba5836fa7f9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.171834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.172461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.172746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.173022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.173668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:24.174004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.175433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.177849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:24.186061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.192304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.192304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.192304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.192569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 429/858] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.192569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.199980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.201420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:24.213825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:24.214962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:24.215995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:24.218633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 430/859] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:24.219317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.309280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee175a00-63a6-4958-ad3e-4a1f478d501c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.320884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:24.335437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80c6a8-5b34-4672-8fd7-36cf7ba461b9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.344796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.345411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.345693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.345999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.346640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:24.346952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.348846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.354540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 430/860] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.355160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.390604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.391837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.392481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.392954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.403314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.405361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:24.419293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:24.420348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:24.421372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:24.424252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 431/861] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:24.430014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.472503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb58950-6153-44db-9f72-1ff6627bb5e0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.498445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:24.527029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.527728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.528011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.528547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.529261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:24.529573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.531060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.552968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 431/862] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.553942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.573841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bfe7f-f5d4-4b5e-ac3a-1d8115e469ad None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.602141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:24.626459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.628187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.628838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.629502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.630640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:24.631157 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.633750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.658601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 432/863] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.664070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:24.691573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda43ba4-6925-4ead-88f2-87cd691f924b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.741795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.742822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.743572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.744024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.744942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=f0506db33e0e47aaa47a9a3fd9bc2f98) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:24.745430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.747598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:24.756220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 432/864] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/role_assignments?user.id=f0506db33e0e47aaa47a9a3fd9bc2f98&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:24.782282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458925-cb1a-49c5-bfc8-b468a96fc808 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:24.785234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:24.825447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f0a59-6cec-4b96-9357-79a4bef60373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:24.826039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0a59-6cec-4b96-9357-79a4bef60373 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:24.826371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0a59-6cec-4b96-9357-79a4bef60373 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:24.826650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0a59-6cec-4b96-9357-79a4bef60373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:24.908770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f0a59-6cec-4b96-9357-79a4bef60373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:24.910623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f0a59-6cec-4b96-9357-79a4bef60373 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.018048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 433/865] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:44:25.023919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:25.128568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:25.130420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69d8f1-f11a-493d-9146-d4cbe610fbfe None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.130741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69d8f1-f11a-493d-9146-d4cbe610fbfe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.131017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69d8f1-f11a-493d-9146-d4cbe610fbfe None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.132995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acb04b-e85f-4040-b780-d1b98d489b0c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:25.133663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 434/866] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:25.137820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.138136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.138413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.146261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:25.159085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:25.160427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:25.171459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:25.172101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.172381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.172654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.185916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:25.187473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:25.209680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:25.210692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:25.211560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:25.211560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:25.211560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:25.211560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 433/867] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:25.211560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:25.234351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 435/868] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:25.328430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcdf66-7719-4aee-9f0f-9e73c23447a7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:25.365293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:25.365928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.366215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.366495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.367141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:25.367444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:25.369088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.382814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 434/869] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:25.388190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:25.495520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8bbf8-f4e5-4964-b2bf-7d44dd530d2d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:25.514300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:25.534451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:25.535276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:25.536001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.536289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.536602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.538545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:25.544056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:25.545737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:25.558910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:25.559846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:25.560988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 436/870] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:25.567435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:25.627985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29a6f8-b367-45c3-a87a-edc392860853 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:25.676947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fa318-7c5f-4b4d-9593-619a57b025dc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:25.680335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248786a-cbaa-4223-86c9-9319853fe92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:25.680951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248786a-cbaa-4223-86c9-9319853fe92d None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.681230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248786a-cbaa-4223-86c9-9319853fe92d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.681506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248786a-cbaa-4223-86c9-9319853fe92d None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.682198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248786a-cbaa-4223-86c9-9319853fe92d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:25.683573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248786a-cbaa-4223-86c9-9319853fe92d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.713917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:25.714599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.714944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.715226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.716007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:25.716318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:25.717814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.731965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 437/871] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:25.738141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:25.768080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 435/872] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:25 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:25.813705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:25.865862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc0605-49a3-40ca-abea-8dec95dd93ed None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:25.911750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:25.912453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:25.912739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:25.913015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:25.913757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=a910227bf388410196f8b201db25ca1b) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:25.914101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:25.915552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa150d-7520-415e-85af-52c687619d45 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:25.922510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 438/873] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:25 2022] GET /identity/v3/role_assignments?user.id=a910227bf388410196f8b201db25ca1b&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:26.386968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2932-5305-432d-b1ed-36cd24d1ec3a None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:26.387352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2932-5305-432d-b1ed-36cd24d1ec3a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:26.387589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2932-5305-432d-b1ed-36cd24d1ec3a None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:26.389246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 436/874] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:26.394701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78201-5220-4e5f-9c4a-22fe054f40ef None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:26.395573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78201-5220-4e5f-9c4a-22fe054f40ef None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:26.395861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78201-5220-4e5f-9c4a-22fe054f40ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:26.414492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-38f78201-5220-4e5f-9c4a-22fe054f40ef None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:26.415700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-38f78201-5220-4e5f-9c4a-22fe054f40ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:26.490725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 439/875] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:26 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:26.766899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d2932-5305-432d-b1ed-36cd24d1ec3a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:26.813885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8879234-a22f-4bab-a604-ebf4672d7d7b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:26.814451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8879234-a22f-4bab-a604-ebf4672d7d7b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:26.814451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8879234-a22f-4bab-a604-ebf4672d7d7b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:26.816030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 440/876] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:26.821879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:26.822204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:26.822451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:26.841057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:26.842425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:26.853718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:26.888066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:26.901293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d2932-5305-432d-b1ed-36cd24d1ec3a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:26.918680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 441/877] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:26 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:26.936963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:26.937979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:26.938519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:26.939021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:26.939939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:26.941749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:27.045182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 437/878] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:26 2022] POST /identity/v3/services => generated 269 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:27.195327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:27.299111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8791b8-b926-441f-8a26-e9f8d74fb13e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:27.339818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:27.340449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.340725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.340995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.341573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:27.342945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:27.352380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:27.353422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:27.354294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 442/879] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:27.359651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:27.471245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b1235-c9ec-4276-8f3d-6d1a4049c372 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:27.502234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:27.508795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:27.509417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.509703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.510008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.510650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:27.510959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:27.512344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e43137-6528-4641-8f81-9226a44a6ba6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:27.517495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 438/880] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:27.523527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:27.563461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:27.646740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4157f-b45e-44a3-8f24-2330289b3ec7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:27.684537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:27.685156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.685442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.685720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.686393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:27.686702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:27.688067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f54e5-2ada-4b76-b767-64037e82958b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:27.693772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 443/881] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:27.887415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4023f-9781-43c9-9180-33487b45739b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.887925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4023f-9781-43c9-9180-33487b45739b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.887925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4023f-9781-43c9-9180-33487b45739b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.889480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 439/882] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:27.894632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.895657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.895855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.914912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:27.916122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:27.957570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c039-0b63-4c53-9932-f541b4e4d2a1 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.958110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c039-0b63-4c53-9932-f541b4e4d2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.958537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c039-0b63-4c53-9932-f541b4e4d2a1 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.961128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 440/883] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:27.965257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:27.965541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:27.965815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:27.986105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:27.988024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:28.000033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 444/884] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:28.070014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 441/885] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:28.280719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.311158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:28.350949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.395073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6084f-16ca-4bca-a0fe-6a372fcf1ae3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.432337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.432975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.433259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.433537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.434149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:28.435556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:28.446055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:28.447079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:28.453509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 445/886] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:28.454348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.492231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d72f-ecc1-42c5-b9d5-72a15566aa36 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.518723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:28.529850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.535422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.535422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.535422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.542061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:28.546911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:28.547864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:28.548780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:28.553874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 442/887] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:28.555245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.581338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:28.594760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce35be-4589-43b0-b92c-000f24ac08a6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.633423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.634078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.634367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.634645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.635665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:28.635665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:28.636941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:28.647249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 446/888] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:28.648154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.668641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be101-6ba9-47ff-9198-8cb12d734f27 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.705834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.706849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.707603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.708083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.709058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:28.709621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:28.711287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:28.724004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 443/889] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:28.733933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.763102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f0925-54fa-4aca-bd1b-66d10245d0a7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.779405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:28.805165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.806089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.806517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.806954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.807732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:28.808186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:28.809733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:28.817440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 447/890] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:28.825954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.842173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d42d73-aed0-4560-b780-9ce16773fb0f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.850743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:28.881304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.882311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.882781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.883266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.895502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:28.898011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:28.915035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:28.916055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:28.916960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:28.916960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:28.916960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:28.923668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 444/891] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:28.925284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:28.945451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f176a6a-9888-4eb3-9af9-a1074a0f0741 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:28.987384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:28.988093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:28.988531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:28.988955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:28.989728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] RBAC: Adding query filter params (interface=public, service_id=3bfaebc50dc34f189310d37ae3281e05, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:28.990221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:28.991771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:28.998226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 448/892] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/endpoints?service_id=3bfaebc50dc34f189310d37ae3281e05&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:29.003357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.037723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:29.042396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93a2e4-dac1-4755-9203-2b68aacd15fc None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.079375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.080406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.080954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.081543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.083162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:29.083992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.085763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.105668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 445/893] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:29.107451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.119526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:29.138195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24783a2-ddcc-4cbc-a6ca-766051d0501b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.182084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f48be-ef62-48a4-bd60-e1748a4e1533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.182707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f48be-ef62-48a4-bd60-e1748a4e1533 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.183241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f48be-ef62-48a4-bd60-e1748a4e1533 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.183523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f48be-ef62-48a4-bd60-e1748a4e1533 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.184551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f48be-ef62-48a4-bd60-e1748a4e1533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.186261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f48be-ef62-48a4-bd60-e1748a4e1533 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.194985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 449/894] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/services => generated 1268 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:44:29.196633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a48325-bc87-4ef5-9698-8a5d09dfaf4b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.197157 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a48325-bc87-4ef5-9698-8a5d09dfaf4b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.197665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a48325-bc87-4ef5-9698-8a5d09dfaf4b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.199382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 450/895] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:29.206472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.207100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.208180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.229400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:29.231103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:29.234031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc54ce-7cdb-4de7-af46-43cb7e5a04bd None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.277446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.278133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.278413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.278680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.287969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.289476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:29.303053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:29.304109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:29.305042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:29.309070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 446/896] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:29.310410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.320120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 451/897] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:29.339101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:29.437678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c4265-a738-463e-a1a4-9764b8b0abff None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.475217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.476239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.476731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.477215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.478201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:29.478738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.480402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.500125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 447/898] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:29.507230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.594012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:29.607684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.619655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434e5e-2f36-45a0-ab15-d9ef0809ffa0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.658168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad85c02-1afc-4ee9-b17b-9b640706946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.658784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad85c02-1afc-4ee9-b17b-9b640706946e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.659064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad85c02-1afc-4ee9-b17b-9b640706946e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.659335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad85c02-1afc-4ee9-b17b-9b640706946e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.701350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:29.726646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfea7e-a2e0-4d15-8a1e-11178f7a694b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.740055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad85c02-1afc-4ee9-b17b-9b640706946e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.742289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad85c02-1afc-4ee9-b17b-9b640706946e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.767249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.767990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.768378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.768661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.769244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.771593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-870c924d-b8ea-4aca-a9d9-e51606029a5b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:29.781188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:29.782159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:29.783038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:29.787704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 448/899] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:29.788920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd0b39-e2a5-4901-901a-428b5e6849ce None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.789203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd0b39-e2a5-4901-901a-428b5e6849ce None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.789477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd0b39-e2a5-4901-901a-428b5e6849ce None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.793960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 449/900] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:29.793960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd0b39-e2a5-4901-901a-428b5e6849ce None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.830225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:29.855604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 452/901] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:29.857247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.857700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.858171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.879064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:29.880823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:29.912869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd0b39-e2a5-4901-901a-428b5e6849ce None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:29.949759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:29.950401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:29.950681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:29.950961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:29.951595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:29.951906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:29.953271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:29.956311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 453/902] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:29.958766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.960686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 450/903] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:29.967491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:29.971475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:30.077559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71736631-5d28-49b5-8865-1ddd8b04f268 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.088540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:30.093707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbfd68-b5d4-4a96-823c-63ff55387ef0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.118240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.118969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.119350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.119723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.134807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.136449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.138005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.138965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.139753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.140186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.140974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:30.141432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.142959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.156350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 451/904] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:30.158231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:30.163573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:30.164773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:30.165871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:30.165871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:30.165871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:30.166848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 454/905] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:30.172178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:30.245515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:30.285208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b09bc-0cb0-4aa8-9efd-7e4c7a920503 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.292462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcb1d2-c23c-4015-abe0-047eae553f55 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.323381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.324171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.324451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.324723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.325428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] RBAC: Adding query filter params (interface=public, service_id=4e9fd9cfd46243dd9dd162bcb7ae42af, region_id=RegionOne) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:30.325735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.327134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.329369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.330438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.330722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.331160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.331809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:30.332184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.338193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 452/906] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:30 2022] GET /identity/v3/endpoints?service_id=4e9fd9cfd46243dd9dd162bcb7ae42af&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:30.338193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:30.338426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.350849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 455/907] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:30.351675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:30.376705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:30.452909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77a8e-3ee7-4eff-9e34-5c3257703666 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.465622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87ffe-c3fb-4f7f-97e7-d6c5d137402c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.483075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:30.491017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.491631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.491912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.492186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.492839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.494509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.502415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.503020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.503301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.503574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.504710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.506170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.511714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 456/908] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:30 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:44:30.512488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:30.573812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 453/909] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:30 2022] POST /identity/v3/services => generated 241 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:30.630854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:30.651815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6e96c-fea8-4739-b1c2-760723fffc41 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.703273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.703928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.704211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.704485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.713442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.715008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:30.729457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:30.730538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:30.732114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 457/910] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:30.736727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:30.846881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbc242-ca05-410a-9e0b-4864dc0f4572 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:30.884247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:30.884784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:30.885067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:30.885407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:30.886145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:30.886451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:30.887880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e11a7-3388-4edb-a47d-e630f43a514d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:30.902106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 454/911] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:30.908141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:31.018641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c910ce-e555-4674-8392-91eee8938ff4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:31.056100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:31.057070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:31.057562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:31.058315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:31.059276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] RBAC: Adding query filter params (role.id=dc7e827817764fa2ba7792621216b8d6, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=a910227bf388410196f8b201db25ca1b) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:31.059794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:31.061418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8cb9a-b46d-408b-9cdf-10c52c794763 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:31.068321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 458/912] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:30 2022] GET /identity/v3/role_assignments?user.id=a910227bf388410196f8b201db25ca1b&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=dc7e827817764fa2ba7792621216b8d6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:31.245887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054503a-ea1c-4379-b5f6-742a1a2f5b59 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:31.246432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054503a-ea1c-4379-b5f6-742a1a2f5b59 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:31.246513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054503a-ea1c-4379-b5f6-742a1a2f5b59 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:31.248020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 455/913] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:31.253372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:31.253669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:31.254045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:31.261368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:31.272639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:31.273838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:31.344439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 459/914] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:31.618228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054503a-ea1c-4379-b5f6-742a1a2f5b59 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:31.675876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:31.732753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054503a-ea1c-4379-b5f6-742a1a2f5b59 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:31.771491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:31.772182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:31.772474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:31.772754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:31.773345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:31.774817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:31.784162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:31.786098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:31.787885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 456/915] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:31.792142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:31.898879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc09cad-8889-4da2-9e8e-1e4f70e2da7b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:31.933778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:31.934423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:31.934709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:31.934986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:31.935627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:31.935935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:31.937303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:31.938462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:31.943358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 460/916] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:31.948972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:32.056431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb6707-f3c6-4acf-8ff6-5bb79cea6064 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:32.074947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:32.092926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:32.093605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:32.093942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:32.094223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:32.095003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:32.095314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:32.096690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9ff7-4e4b-4e7d-a1ef-6e856c0e12e2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:32.102483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 457/917] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 13:44:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:32.108373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:32.221850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2c9c4-66ee-4cc6-b076-8a6be5fb66fd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:32.260737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254cd60-10d4-4b18-97a0-c36075de7e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:32.261359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254cd60-10d4-4b18-97a0-c36075de7e26 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:32.261754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254cd60-10d4-4b18-97a0-c36075de7e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:32.262138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254cd60-10d4-4b18-97a0-c36075de7e26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:32.262920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254cd60-10d4-4b18-97a0-c36075de7e26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:32.264295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254cd60-10d4-4b18-97a0-c36075de7e26 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:32.357507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 461/918] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:32.589253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d60d97-d882-4e45-966b-323de009dac7 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:32.590328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d60d97-d882-4e45-966b-323de009dac7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:32.590822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d60d97-d882-4e45-966b-323de009dac7 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:32.592628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 458/919] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:32.598652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8eacf-015d-4fd8-824b-79aafdc37fb7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:32.599234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8eacf-015d-4fd8-824b-79aafdc37fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:32.599738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8eacf-015d-4fd8-824b-79aafdc37fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:32.620200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-39c8eacf-015d-4fd8-824b-79aafdc37fb7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:32.621760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-39c8eacf-015d-4fd8-824b-79aafdc37fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:32.677375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:32.702564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 462/920] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:32.784506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa1181-c3c9-4e18-be83-fafd60c981c9 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:32.784843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa1181-c3c9-4e18-be83-fafd60c981c9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:32.785600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa1181-c3c9-4e18-be83-fafd60c981c9 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:32.787294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 459/921] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:32.792693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704ec43-8a9d-4b7d-85e6-90c0d2424262 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:32.793038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704ec43-8a9d-4b7d-85e6-90c0d2424262 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:32.793264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704ec43-8a9d-4b7d-85e6-90c0d2424262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:32.812651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7704ec43-8a9d-4b7d-85e6-90c0d2424262 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:32.813936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7704ec43-8a9d-4b7d-85e6-90c0d2424262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:32.887759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 463/922] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:32.982588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa1181-c3c9-4e18-be83-fafd60c981c9 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.053694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde3cb-aacd-4569-8297-7af7ed22f1c6 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.054055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde3cb-aacd-4569-8297-7af7ed22f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.054291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde3cb-aacd-4569-8297-7af7ed22f1c6 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.055900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 464/923] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:33.061201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.061492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.061769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.084039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:33.085488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:33.099103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa1181-c3c9-4e18-be83-fafd60c981c9 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.136537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.137235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.137515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.137930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.138590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.140013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:33.150599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:33.151629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:33.152555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 460/924] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 1 13:44:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:33.159345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.173427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 465/925] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:33.174153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.222131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:33.228393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:33.287209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f33b9-a3ba-436c-acbc-7babb5ee0b10 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.304899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6a63-be4a-42cb-a3fd-b6ab1d95a1ee None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.325200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.325810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.326122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.326414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.327078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:33.327378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.328799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.339172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 461/926] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:33.341364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.357047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.358194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.358725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.358964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.359804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.361307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:33.371508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:33.372764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:33.373811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:33.375793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 466/927] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:33.385069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.476197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f246-4810-4194-8c6b-1f51b3f06146 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.486458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:33.507588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc7380-f1e9-447e-adfe-49df565e53d5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.510045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:33.512852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.513515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.513799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.514109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.514744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:33.515050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.516482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.527445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 462/928] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:33.528337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.548907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.549544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.549840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.550132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.551038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:33.551486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.553527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.559105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 467/929] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:33.560035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.650124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2b315-5314-4745-a32e-6e69e8fcf5be None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.676659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac491f8-9906-4c88-b97d-449a085a101b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.687232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.687871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.688155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.688430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.701399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:33.706298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:33.708329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 463/930] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:33.709203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.724108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.726815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.726815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.726815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.726815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.727369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.747315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:33.816060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65971a-ad26-4aed-800a-1728ffeef256 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:33.843950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 468/931] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:33.846068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:33.877745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:33.878530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:33.880750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:33.880750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:33.880978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:33.881288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:33.882985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:33.886466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:33.894436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 464/932] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:33.895950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.002522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195f2bb-9c1b-4d8d-812f-4862a2bf0864 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.014494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:34.030248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9711fc-1c9d-481c-b606-54fa51896616 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.048841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.050170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.051019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.051756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.052636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:34.053254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.054676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.065439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 469/933] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:34.072081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.081071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.081695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.082128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.082478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.083287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] RBAC: Adding query filter params (interface=public, service_id=fedcce69cab54e88b0cdea3eb4d8dc5d, region_id=RegionOne) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:34.083607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.084993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.095070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 465/934] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:33 2022] GET /identity/v3/endpoints?service_id=fedcce69cab54e88b0cdea3eb4d8dc5d&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:34.098197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.178229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:34.194384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b5238-ee1a-437f-9000-3d3627b713fb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.215592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51cf14-7190-44a6-b322-a5942183d322 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.232633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.233375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.233664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.233964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.245995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.247553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.261797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59b07b-9c73-4746-960b-7621935e0f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.262828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59b07b-9c73-4746-960b-7621935e0f51 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.263334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59b07b-9c73-4746-960b-7621935e0f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.263841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59b07b-9c73-4746-960b-7621935e0f51 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:34.264180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:34.265167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:34.266126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:34.266126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:34.266126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:34.267334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59b07b-9c73-4746-960b-7621935e0f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.268963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59b07b-9c73-4746-960b-7621935e0f51 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.270196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 470/935] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:34.271222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.286327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 466/936] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:34 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:44:34.389344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001642a0-1d9d-4a66-bdcc-15fe08f92e28 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.391577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114da332-1059-4be4-b33b-907d5f167eb5 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.391866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114da332-1059-4be4-b33b-907d5f167eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.392138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114da332-1059-4be4-b33b-907d5f167eb5 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.393741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 467/937] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:34.399686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.399965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.400242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.426980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:34.428589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:34.430456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.431194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.431538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.431884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.432601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:34.432970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.435952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.442946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:34.448409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 471/938] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:34.453864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.456491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:34.528218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 468/939] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:34.571543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb3ea6-0435-4977-9cdb-716295f8ad6f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.608245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.608906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.609225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.609503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.617585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.619446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:34.631977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:34.632966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:34.633944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 472/940] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:44:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:34.639123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.746035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd462f0e-c6e3-4624-a50c-0a3de64224d9 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.787880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.788410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.788696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.788977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.789717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:34.790064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.791592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.796213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.811200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 469/941] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 13:44:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:34.816802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:34.829151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:34.868453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:34.911249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7308a42-8128-4444-a0ce-2cbc9b840530 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.945296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4198-a3c5-43d8-944f-54391a7a6ba5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:34.946657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.947490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.948010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.948285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.948931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:34.950599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:34.953043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING keystone.server.flask.common [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 13:44:34.986458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:34.987101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:34.987383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:34.987660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:34.999248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.000721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.003098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.sql.core [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 13:44:35.003098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 13:44:35.003098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [parameters: {'id': 'f67c2f91e36b4a0a80a672e79e9a651b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 13:44:35.003098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81706) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f67c2f91e36b4a0a80a672e79e9a651b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f67c2f91e36b4a0a80a672e79e9a651b. Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:44:35.011702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 13:44:35.012733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application yield resource Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise self.value Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application session.commit() Sep 01 13:44:35.013919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.flush() Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application compat.raise_( Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 13:44:35.015031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 13:44:35.016138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:44:35.017125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [parameters: {'id': 'f67c2f91e36b4a0a80a672e79e9a651b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.018229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:35.019454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f67c2f91e36b4a0a80a672e79e9a651b. Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.026159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 473/942] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 1 13:44:34 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:35.027358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:35.027850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:35.028734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:35.029648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:35.029648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:35.029648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:35.031932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 470/943] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:35.046018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:35.107116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:35.144633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17340d-517b-4753-a171-21ee92420a70 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:35.153543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2235-a02a-4547-94c2-a879be580e61 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:35.160721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:35.183205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:35.183911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.184213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.184572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.193005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:35.193189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.194232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.194720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.195173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.195285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.196559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:35.197065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.198622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:35.209167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:35.210119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:35.211009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:35.215391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 474/944] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:35.220581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:35.221109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 471/945] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:35.222322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:35.326504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99157cb-6c56-43b5-a62a-21510d0db835 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:35.333503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74b74c-be9d-4ab0-bd61-37a94bbb1364 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:35.356810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:35.365318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:35.366540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.366540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.367112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.368282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:35.368794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.370841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.372844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:35.373587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.373868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.374182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.381517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.383014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:35.396106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:35.397069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:35.397977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:35.405953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 472/947] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:35.405953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:35.406290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 475/947] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:35.525496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba375b-9d16-411b-b0b6-94753fde05c8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:35.565190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:35.565854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.566723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.567051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.567960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:35.568271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.569702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8b03f-6d47-4565-a989-a566f5a172a2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.584498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 473/948] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:35.590249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:35.634758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:35.710187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035246-e6c2-4307-a57f-1ebcaac26d2e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:35.747101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcaf60-4fd9-4c96-a0d6-16e3f5e5e5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:35.748162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcaf60-4fd9-4c96-a0d6-16e3f5e5e5b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.748646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcaf60-4fd9-4c96-a0d6-16e3f5e5e5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.749139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcaf60-4fd9-4c96-a0d6-16e3f5e5e5b4 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/a910227bf388410196f8b201db25ca1b/roles/dc7e827817764fa2ba7792621216b8d6` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.751601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcaf60-4fd9-4c96-a0d6-16e3f5e5e5b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=a910227bf388410196f8b201db25ca1b, role_id=dc7e827817764fa2ba7792621216b8d6)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:35.753218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcaf60-4fd9-4c96-a0d6-16e3f5e5e5b4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:35.815480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 476/949] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:44:35 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/a910227bf388410196f8b201db25ca1b/roles/dc7e827817764fa2ba7792621216b8d6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:44:35.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a54b-9d29-41eb-9ede-ce54c190caca None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.955384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a54b-9d29-41eb-9ede-ce54c190caca None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.955384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a54b-9d29-41eb-9ede-ce54c190caca None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.957501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 474/950] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:35.963263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:35.963549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:35.963828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:35.983958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:35.985494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:36.062023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 477/951] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:36.336239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11f81a-d262-45db-be37-247868cfa670 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:36.336644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11f81a-d262-45db-be37-247868cfa670 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:36.336889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11f81a-d262-45db-be37-247868cfa670 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:36.338423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 475/952] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:36.339295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:36.343350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:36.343646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:36.343905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:36.362758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:36.364033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:36.366776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:36.415286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:36.458549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69ebf-c3c5-497c-9d7b-570b4b4461f5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:36.461988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 476/953] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:36.502517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:36.503151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:36.503937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:36.504222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:36.504810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:36.506190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:36.515664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:36.516818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:36.518181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 478/954] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:36.525524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:36.633701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbeafa-87f2-403c-87c6-c5e2fe330da0 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:36.671974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:36.672591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:36.672864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:36.673133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:36.673764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:36.674089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:36.675691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:36.681033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 477/955] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:36.686900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:36.756465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:36.805607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:36.818930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:36.834899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7506e-56cd-4759-b3cf-6895903caed9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:36.884557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:36.885090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:36.885372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:36.885683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:36.886241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeb964-fa24-4af7-8212-2fc433ee9476 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:36.886355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:36.886612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:36.888076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:36.893443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 479/956] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:36 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:36.936059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:36.936716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:36.936991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:36.937262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:36.937856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:36.939461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:36.950251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:36.951433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:36.952265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 478/957] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:36.958527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:37.078454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc205a8-0362-4d0f-823a-6d86af7200f2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:37.116060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:37.116684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.116963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.117237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.118182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:37.118485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:37.119846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:37.125776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 480/958] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:37.132308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:37.236139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:37.244285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816c40f-024b-42a3-b976-69e1f06c07d6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:37.281011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:37.281694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.282010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.282288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.282933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:37.283237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:37.284639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fd0c2-38ae-4c6d-a2aa-1b1748c0523f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:37.295124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 479/959] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:37.296459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:37.317675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:37.410425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fdd29-3ae3-42da-832c-964286386ba6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:37.451079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73726988-3910-46ba-a8c3-bcef320c8f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:37.451754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73726988-3910-46ba-a8c3-bcef320c8f19 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.452082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73726988-3910-46ba-a8c3-bcef320c8f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.452362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73726988-3910-46ba-a8c3-bcef320c8f19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.453038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73726988-3910-46ba-a8c3-bcef320c8f19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:37.454562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73726988-3910-46ba-a8c3-bcef320c8f19 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:37.529374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12048f-46df-490a-9b13-ad09faa64624 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.529686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12048f-46df-490a-9b13-ad09faa64624 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.530016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 481/960] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:37.530136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12048f-46df-490a-9b13-ad09faa64624 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.531798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 480/961] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:37.537652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc006427-35fc-4ca4-aaf9-9268544db20e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.538115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc006427-35fc-4ca4-aaf9-9268544db20e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.538395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc006427-35fc-4ca4-aaf9-9268544db20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.561519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-dc006427-35fc-4ca4-aaf9-9268544db20e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:37.563196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dc006427-35fc-4ca4-aaf9-9268544db20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:37.661256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:37.668785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 482/962] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:37.887806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41dfb9-999c-4bcc-bda4-2606c3f1da80 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.888117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41dfb9-999c-4bcc-bda4-2606c3f1da80 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.888307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41dfb9-999c-4bcc-bda4-2606c3f1da80 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.889865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 481/963] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:37.895925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:37.896211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:37.896488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:37.915156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:37.916373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:37.951967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41dfb9-999c-4bcc-bda4-2606c3f1da80 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:37.998833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 483/964] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:38.066960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41dfb9-999c-4bcc-bda4-2606c3f1da80 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:38.104085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:38.105094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.105588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.106096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.106963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:38.108708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:38.111513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING keystone.server.flask.common [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 13:44:38.154695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.sql.core [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 13:44:38.154695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 13:44:38.154695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [parameters: {'user_id': '4718ad60c1074e1cb30dd04f0bead840', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 13:44:38.154695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81705) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application result.read() Sep 01 13:44:38.173624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:44:38.174631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application yield resource Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise self.value Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application session.commit() Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 13:44:38.175625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self.session.flush() Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application compat.raise_( Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 13:44:38.176535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:44:38.178735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application result.read() Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:44:38.180835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [parameters: {'user_id': '4718ad60c1074e1cb30dd04f0bead840', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:38.182947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 13:44:38.185017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 13:44:38.187652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 13:44:38.187652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 13:44:38.187652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 13:44:38.187652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 13:44:38.187652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 13:44:38.187652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:38.194429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 482/965] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 13:44:37 2022] POST /identity/v3/users => generated 160 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:38.198739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:38.305718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:38.312363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87fc51-a15e-43aa-b77b-b6aba9cd9d83 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:38.339553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:38.350248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:38.360387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:38.361085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.361377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.361657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.372432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:38.374639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:38.391490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:38.392693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:44:38.393626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:38.397754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 484/966] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:38.399016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:38.448575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c3e8d-2af5-4fd7-86e8-8e243a82fa98 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:38.495436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:38.496112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.496395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.496681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.512115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:38.518683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:38.521162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:38.522593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:38.529939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 483/967] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 13:44:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:38.529939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:38.534133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e9627-6c16-4f7e-8e60-fe2d4c3e7084 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:38.577090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:38.577090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.577090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.577090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.586673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:38.586673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:38.586673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:38.619038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 485/968] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 13:44:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:38.635741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:38.658416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595499-6a0d-4ebc-b102-cf464bd59b20 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:38.702038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:38.702584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.702931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.703211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.703894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:38.704239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:38.705828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:38.720962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 484/969] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:44:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:38.727383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:38.793515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:38.852727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003dd82-6c67-4d70-b4ab-f4fbe5685681 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:38.889325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:38.889998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.890288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.890566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.903784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:38.905313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c0a4cb35-b8f0-4979-b4e7-f0520af63f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:38.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:38.922851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:38.924477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:38.924477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:38.924477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:38.924477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 486/970] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:38.929194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:38.949924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45543-4476-4ca6-a028-09d4eadf9a03 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.950216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45543-4476-4ca6-a028-09d4eadf9a03 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.950491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45543-4476-4ca6-a028-09d4eadf9a03 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.951994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 487/971] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:38.957406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:38.957608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:38.957891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:38.977118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:38.978386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:39.042388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e17ffe-ef87-4cbb-a81e-4afafe20b88c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:39.061954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 488/972] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:39.083997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:39.084644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.084929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.085199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.085843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:39.086181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:39.087757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:39.105338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 485/973] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:39.107012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:39.138247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:39.215566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9bc0c-b333-4491-b15d-09f50fc9af42 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:39.255252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:39.255894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.256176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.256452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.263720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:39.265143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:39.276724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:39.277823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:39.279242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 489/974] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:44:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:39.283663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:39.293638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:39.330443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:39.395359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e108d-06f8-4020-8eed-1b1870b473f2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:39.427412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:39.432541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:39.433370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.433822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.434284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.435204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:39.435667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:39.437253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:39.439500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33d6e3-ced3-4142-8b83-4e883a9d635f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:39.461456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 486/975] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 13:44:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:39.462720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:39.478768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b98c9-8801-4d64-be29-9d39487849f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:39.479393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b98c9-8801-4d64-be29-9d39487849f0 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.479675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b98c9-8801-4d64-be29-9d39487849f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.479952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b98c9-8801-4d64-be29-9d39487849f0 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.480603 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b98c9-8801-4d64-be29-9d39487849f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:39.482013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b98c9-8801-4d64-be29-9d39487849f0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:39.529288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:39.553934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 490/976] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:39 2022] POST /identity/v3/services => generated 242 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:39.554602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd842-88f0-4777-b526-9b8d800c5420 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.554602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd842-88f0-4777-b526-9b8d800c5420 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.554602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd842-88f0-4777-b526-9b8d800c5420 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.556126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 491/977] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:39.561608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.561979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.562228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.569727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d78347-35c3-4913-809e-ba30ef81a472 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:39.585248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:39.587021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:39.623551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:39.624173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.624458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.624735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.670859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 492/978] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:39.706237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:39.708331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:39.774952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:39.818149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 487/979] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:39 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:39.824140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:39.906284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:39.928512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baec5d0-56c5-43b7-9bcd-c04f6b79b85a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:39.948745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:39.966454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:39.967200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:39.967939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:39.968530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:39.980387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:39.981838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:39.998511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:39.999519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:40.000475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:40.000475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:40.000475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:40.000475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 493/980] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:40.006420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.065343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0aa02-8d59-4206-a033-e5de94989c1f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.105539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.106492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.106943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.107422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.108292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.109838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.110816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e05f-cca6-4fd7-b532-27837a11f128 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:40.120159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:40.121354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:40.122203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 488/981] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:40.127229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.148399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:40.155669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:40.155949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.156980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.161506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.161506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.161506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:40.161506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.161506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.173087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 494/982] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:40.178816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.251728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec8e1-5510-4621-b737-42a596a7fb33 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.288575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.289236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.289491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.289716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.290104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1cd42-6adf-4cb5-9320-299f3eb19161 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.290469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:40.290772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.293037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.297588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 489/983] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:40.303006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.330452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.331014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.331830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.332113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.339139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.340566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:40.351877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:40.352801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:40.353817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 495/984] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:40.359481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.402039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:40.415229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509d13f-c970-4365-96f1-520cc953a5b0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.416268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:40.452155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.452895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.453127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.453405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.454075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:40.454399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.455762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.460985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 490/985] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:40.466757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.480443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b12bb-68e1-40ae-ae3c-20a9072d5eeb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.522199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.522863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.523259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.523534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.524227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:40.524539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.525982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.543381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 496/986] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:40.544879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.587772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc035031-adcb-46a9-bf4e-1bfa14807987 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.632487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.633119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.633414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.633692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.634447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] RBAC: Adding query filter params (interface=admin, service_id=fedcce69cab54e88b0cdea3eb4d8dc5d, region_id=RegionOne) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:40.634768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.636154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.646324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 491/987] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/endpoints?service_id=fedcce69cab54e88b0cdea3eb4d8dc5d&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:40.647433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:40.669174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4d472-6771-4140-aaac-f882d3213a8d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.669743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:40.681764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:40.707388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.708007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.708724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.709014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.710115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] RBAC: Adding query filter params (role.id=dc7e827817764fa2ba7792621216b8d6, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=a910227bf388410196f8b201db25ca1b) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:40.710433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.712353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f5e46-4bb5-4367-a639-ec1eb4d80fca None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.718408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 497/988] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/role_assignments?user.id=a910227bf388410196f8b201db25ca1b&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=dc7e827817764fa2ba7792621216b8d6 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:40.771972 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d132e-ab64-42de-b5dc-b106e55e11b1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:40.810998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e098e-792b-498d-a14c-51de48a63da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:40.811940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e098e-792b-498d-a14c-51de48a63da7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:40.812404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e098e-792b-498d-a14c-51de48a63da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:40.812863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e098e-792b-498d-a14c-51de48a63da7 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:40.813661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e098e-792b-498d-a14c-51de48a63da7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:40.815290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e098e-792b-498d-a14c-51de48a63da7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:40.823712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 492/989] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:40 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:44:41.512101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90d22b-b8e6-44f0-9227-97078e5301f2 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:41.512481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90d22b-b8e6-44f0-9227-97078e5301f2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:41.512663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90d22b-b8e6-44f0-9227-97078e5301f2 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:41.514177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 498/990] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:41.519353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:41.519800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:41.519967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:41.538673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:41.540188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:41.613033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 493/991] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:41.888225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90d22b-b8e6-44f0-9227-97078e5301f2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:41.994553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90d22b-b8e6-44f0-9227-97078e5301f2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:42.029385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:42.030038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.030321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.030595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.031172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:42.032516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:42.041233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:42.042297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:42.043135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:42.044493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 499/992] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:42.048187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:42.060377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:42.152265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb71b-1d30-4c11-8a1b-00df5d57b4c8 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:42.192633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:42.193242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.193518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.193884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.194447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:42.194749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:42.196143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:42.201516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 494/993] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:42.207632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:42.248247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:42.310475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d68041-efd3-42e1-ae83-bd81d5e8a6b1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:42.345765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:42.346397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.347205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.347482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.348156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:42.348462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:42.349807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:42.359669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 500/994] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:42.360994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:42.438204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:42.464206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6596822-9545-48e1-a55e-8db6d87f58b9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:42.500079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:42.500684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.500957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.501226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.501951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] RBAC: Adding query filter params (interface=public, service_id=bc8f465c84ed448c944a512e3866e3c4, region_id=RegionOne) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:42.502265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:42.503722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018f341-5093-41f7-8fc7-c515ecd0f84b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:42.513591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 495/995] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:42 2022] GET /identity/v3/endpoints?service_id=bc8f465c84ed448c944a512e3866e3c4&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:42.514585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:42.624760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9ee79-cfcd-4650-9de1-0ed775141287 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:42.626352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:42.661368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06325ed0-3db4-4555-9397-9154d0b06623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:42.662011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06325ed0-3db4-4555-9397-9154d0b06623 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.662286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06325ed0-3db4-4555-9397-9154d0b06623 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.662555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06325ed0-3db4-4555-9397-9154d0b06623 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.663156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06325ed0-3db4-4555-9397-9154d0b06623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:42.664500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06325ed0-3db4-4555-9397-9154d0b06623 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:42.677946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 501/996] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:42 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 13:44:42.690847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799f507-94fe-443f-bdab-baac4bb61a71 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.691375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799f507-94fe-443f-bdab-baac4bb61a71 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.691375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799f507-94fe-443f-bdab-baac4bb61a71 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.692802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 496/997] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:42.699104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.699300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.699525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.720978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:42.720978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:42.781240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba5793-0e89-429e-ad3d-a22c64984073 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.781526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba5793-0e89-429e-ad3d-a22c64984073 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.781802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba5793-0e89-429e-ad3d-a22c64984073 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.783435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 497/998] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:42.789273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:42.789710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:42.790044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:42.802733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 502/999] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:42.810668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:42.811882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:42.885981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 498/1000] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:43.074076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.155004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.177401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:43.183405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897e6ad-5885-46b9-b418-a2dee1e41904 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.222516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.223262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.223548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.223820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.224403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.225769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.230216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:43.237257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:43.242667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:43.243500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 503/1001] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 1 13:44:43 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:43.245221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.270787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baf355-e868-4bb2-9895-7463eacd3d41 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.311336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.311971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.312255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.312533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.313123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.314662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:43.324363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:43.325370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:43.326255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 499/1002] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 13:44:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:43.332143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.355816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef34ff-63cd-45a4-8943-febe94e7edc4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.393956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.394936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.395460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.396182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.396971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] RBAC: Adding query filter params (name=image) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:43.397429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.399097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.408966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 504/1003] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:43.410158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.435026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf5a7-7dc9-401c-8971-7a6077c0fca5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.436128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:43.469704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:43.474580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.475193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.475506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.475783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.476486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:43.476793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.478203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.483458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 500/1004] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:43.489341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.527957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b2232-a2e2-429d-915c-9a4d8a5e2947 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.564677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.565293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.566211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.566478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.567111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] RBAC: Adding query filter params (type=image) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:43.567418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.568768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.582202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 505/1005] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:43.608333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43cfcd-1efc-4c58-9874-27c166726ac4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.648712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.649376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.649627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.649934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.650587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:43.650895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.652301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef43eb5-1b9f-422c-9d40-13b967479bf9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.658083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 501/1006] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 13:44:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:43.664516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:43.788833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:43.801643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45eda-1655-4534-a058-c5005b2a460a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:43.839051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89f557-190b-4ed1-a394-5f6ffd5d74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:43.839585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89f557-190b-4ed1-a394-5f6ffd5d74c9 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:43.839868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89f557-190b-4ed1-a394-5f6ffd5d74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:43.840142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89f557-190b-4ed1-a394-5f6ffd5d74c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:43.840812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89f557-190b-4ed1-a394-5f6ffd5d74c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:43.842202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89f557-190b-4ed1-a394-5f6ffd5d74c9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:43.906468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 506/1007] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:44:44.599985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea12bc-b85f-4ead-8158-2b1cebd77968 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:44.600503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea12bc-b85f-4ead-8158-2b1cebd77968 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:44.600503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea12bc-b85f-4ead-8158-2b1cebd77968 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:44.602263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 502/1008] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:44.607414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:44.607805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:44.607987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:44.626608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:44.628336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:44.705736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 507/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:44.974701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea12bc-b85f-4ead-8158-2b1cebd77968 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:45.040177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:45.083269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea12bc-b85f-4ead-8158-2b1cebd77968 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:45.118190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:45.118813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.119089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.119359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.119934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:45.121346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:45.130751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:45.131772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:44:45.132629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:45.139861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 503/1010] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:45.140013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:45.219606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:45.246015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfca5e-7825-4435-b397-c77440330bdb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:45.280623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:45.281679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.282202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.282714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.283745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:45.284812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:45.286720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:45.292427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 508/1011] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:45.297785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:45.405547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15be914-3242-4e97-96ba-283f3292a671 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:45.429525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:45.440996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:45.441627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.441956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.442225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.442820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:45.443118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:45.444474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa0537-bcf8-41fe-bf33-8325196c418b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:45.454158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 504/1012] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 13:44:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:45.455328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:45.526179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7daa01-54a6-4b24-ad8c-f1e2a096eab8 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.526488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7daa01-54a6-4b24-ad8c-f1e2a096eab8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.526662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7daa01-54a6-4b24-ad8c-f1e2a096eab8 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.528124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 505/1013] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:45.533025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e93a4-57cb-468f-9998-960a63b04285 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.533312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e93a4-57cb-468f-9998-960a63b04285 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.533586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e93a4-57cb-468f-9998-960a63b04285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.552783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6a8e93a4-57cb-468f-9998-960a63b04285 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:45.554010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6a8e93a4-57cb-468f-9998-960a63b04285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:45.560736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2466b5-78d9-49c2-a88f-ebd6a6af97c5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:45.610213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b372364-6cca-477c-9f4f-462399fd6f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:45.610951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b372364-6cca-477c-9f4f-462399fd6f91 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.611180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b372364-6cca-477c-9f4f-462399fd6f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.611456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b372364-6cca-477c-9f4f-462399fd6f91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.612836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b372364-6cca-477c-9f4f-462399fd6f91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:45.614366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b372364-6cca-477c-9f4f-462399fd6f91 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:45.624054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:45.628329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 506/1014] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:45.689372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 509/1015] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:45.867841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e710-7f5a-4b48-a624-729a9209a150 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.868201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e710-7f5a-4b48-a624-729a9209a150 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.868486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e710-7f5a-4b48-a624-729a9209a150 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.870176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 507/1016] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:45.875565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:45.875861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:45.876131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:45.895802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:45.897035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:45.903637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e710-7f5a-4b48-a624-729a9209a150 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:45.981656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 510/1017] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:45 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:46.025065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e710-7f5a-4b48-a624-729a9209a150 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:46.044450 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:46.062277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:46.062906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:46.063179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:46.063467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:46.064178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:46.066367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:46.189766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 508/1018] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:45 2022] POST /identity/v3/services => generated 242 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:44:46.256677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:46.368446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd1fb-6dc6-406e-a168-acfc791e2b3f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:46.390185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:46.413200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae81ac-efe7-4bfc-881a-dd80d7a0dca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:46.413840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae81ac-efe7-4bfc-881a-dd80d7a0dca9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:46.414187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae81ac-efe7-4bfc-881a-dd80d7a0dca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:46.414642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae81ac-efe7-4bfc-881a-dd80d7a0dca9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:46.415846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae81ac-efe7-4bfc-881a-dd80d7a0dca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:46.417306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae81ac-efe7-4bfc-881a-dd80d7a0dca9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:46.418915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 511/1019] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:46.424858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:46.537345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a727735-30c4-4110-b5fc-09e9874a9a72 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:46.573158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c3fb-08f1-4d74-8017-009a45d223f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:46.574514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c3fb-08f1-4d74-8017-009a45d223f7 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:46.574514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c3fb-08f1-4d74-8017-009a45d223f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:46.574514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c3fb-08f1-4d74-8017-009a45d223f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:46.577092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832c3fb-08f1-4d74-8017-009a45d223f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:46.577092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832c3fb-08f1-4d74-8017-009a45d223f7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:46.705043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 509/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:44:46 2022] POST /identity/v3/projects => generated 311 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:47.931143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8adafd-9b4f-492b-bbf0-b6282d9a1930 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:47.931664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8adafd-9b4f-492b-bbf0-b6282d9a1930 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:47.931664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8adafd-9b4f-492b-bbf0-b6282d9a1930 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:47.933022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 512/1021] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:47.939072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926a7a-4763-4efb-bb43-e1671a8dd73e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:47.939648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926a7a-4763-4efb-bb43-e1671a8dd73e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:47.939648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926a7a-4763-4efb-bb43-e1671a8dd73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:47.959652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9d926a7a-4763-4efb-bb43-e1671a8dd73e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:47.960860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9d926a7a-4763-4efb-bb43-e1671a8dd73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:47.990224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:48.038464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 510/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:48.314205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8adafd-9b4f-492b-bbf0-b6282d9a1930 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:48.355281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fae5b9-a0c9-464d-97e2-7ad2d0a70295 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.355619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fae5b9-a0c9-464d-97e2-7ad2d0a70295 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.355833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fae5b9-a0c9-464d-97e2-7ad2d0a70295 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.357394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 511/1023] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:48.363194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.363501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.363760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.384086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:48.385284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:48.430473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8adafd-9b4f-492b-bbf0-b6282d9a1930 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:48.471704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:48.472429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 512/1024] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:48.474650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.475315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.475637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.487524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:48.492715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:48.493715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:48.494618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 513/1025] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 13:44:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:48.499899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:48.616456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:48.628090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa71139-7990-49bf-be98-ccf3b5e14f96 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:48.668777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:48.669813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.670324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.670784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.671734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:48.672034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:48.673995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:48.686268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 513/1026] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:44:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:48.691198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:48.744596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:48.753591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:48.809959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8564cf-195b-4387-9d97-a79a8c3340bf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:48.847072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:48.847712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.847996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.848275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.858978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:48.860357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:48.861166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e528-5d34-4b97-81d7-a70a3493b8d6 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9ad5e5dc-c0ff-4c07-b2cc-d3178b38032b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:48.875717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:48.876722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:48.878298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:48.878298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:48.878298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:48.878298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 514/1027] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:48.878774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691c881-d2c0-444c-86af-a6470c7e7bcf None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.879204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691c881-d2c0-444c-86af-a6470c7e7bcf None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.879299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691c881-d2c0-444c-86af-a6470c7e7bcf None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.880678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 515/1028] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:48.883601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691c881-d2c0-444c-86af-a6470c7e7bcf None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:48.901771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:48.902525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.902790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.903064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] PATH_INFO: `/v3/services/image` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.903641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:48.905067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6edb6d03-0865-449e-b301-e4f33201878a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:48.914650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:48.915668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:48.916518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 514/1029] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 1 13:44:48 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:48.917336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:48.917336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:48.917458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:48.935881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:48.937061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:48.948714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:48.989188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:48.996427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691c881-d2c0-444c-86af-a6470c7e7bcf None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.010932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 515/1030] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:49.011780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.035372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.035985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.036272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.036547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.037193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:49.037498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.038906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.050984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 516/1031] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:49.056521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.141635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e24a8-27fe-4de7-b351-5bd3e60a4c49 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.171211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878dc25-d50f-4f26-8492-f86c3386eee5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.181380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.181961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.182270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.182923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.183729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] RBAC: Adding query filter params (name=image) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:49.184037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.185428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.193304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 516/1032] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:49.198983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.209995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.210627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.210908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.211192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.213168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.219442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.220880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:49.232485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:49.233663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:49.234812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 517/1033] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:49.239176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.261380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.311329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0b1c-dd5b-4b28-8dd2-6321ea314fed None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.344975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d6db4-0935-4599-b96a-1fbd9abc5480 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.351424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.352036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.352311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.352581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.353208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] RBAC: Adding query filter params (type=image) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:49.353515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.355027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.359819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 517/1034] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:49.360587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.380556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.381164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.381445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.381726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.382477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:49.382787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.384210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.413128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 518/1035] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:49.413957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.445428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.475826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38d906-d89c-4f1c-b973-909f4206c488 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.500598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.514619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.515314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.515599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.515878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.517001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.519080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.520725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 518/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:49.521621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.527077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b31711-03cc-4103-a5d5-8047bcec477e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.563110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.563723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.564007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.564281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.565040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] RBAC: Adding query filter params (interface=public, service_id=b2fb50bccd904e99ab852591fe554e83, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:49.565421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.566855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.572004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 519/1037] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/endpoints?service_id=b2fb50bccd904e99ab852591fe554e83&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:49.572774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.636949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ca10-fab6-44ab-8b70-3776ee87fa32 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.672561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.673087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.673376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.673473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ee655-81ab-4bd9-8658-29efc47af47a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.673653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.685722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.708036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:49.708641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:49.708926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:49.709204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:49.709852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.711300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.722093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.744484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:49.745973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:49.848291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 519/1038] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:49.849538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.858103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 520/1039] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:44:49 2022] POST /identity/v3/users => generated 285 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:49.859498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:49.909456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:49.975716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5be404-4fae-44b5-ae99-e1335e1129e0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:49.986165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddae0-c518-4297-bee4-492b30765918 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:50.017381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:50.018705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:50.019085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:50.019368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:50.020103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:50.021497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:50.025114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:50.026333 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:50.026619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:50.026898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:50.028180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 520/1040] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:44:50.038746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:50.040236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:50.057051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:50.058172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:50.059057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:50.059057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:50.059057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:50.059057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 521/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:50.064292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:50.186983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0202b0c-3790-4714-b321-2e71bb37da3d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:50.196275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:50.224573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:50.226001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:50.226308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:50.226591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:50.227280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:50.227589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:50.228966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:50.242137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 521/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:50.247866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:50.300969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:50.359979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf77a0-cc07-4dfe-abd5-59802839ad04 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:50.398014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:50.398649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:50.398924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:50.399314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:50.406584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:50.408071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:50.420661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:50.421634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:50.427091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 522/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:50.430662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:50.541441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127843f2-414f-4341-b9b0-971b4f45c40e None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:50.578273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:50.579210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:50.579494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:50.579766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:50.580464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:50.580774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:50.582241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd67a6-e1d7-44a6-84e7-e82de2095c34 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:50.601500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 522/1044] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:50.602891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:50.722860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da84a-5319-4c72-a411-84e824a83eeb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:50.731931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:50.767853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:50.768525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:50.768996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:50.769278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:50.770059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] RBAC: Adding query filter params (role.id=dc7e827817764fa2ba7792621216b8d6, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=cf5c9c7caaf4451a9b1ca1a1095408d1) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:50.770374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:50.771797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f798-19fb-4496-9062-fbe1f90c4251 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:50.777721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 523/1045] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:50 2022] GET /identity/v3/role_assignments?user.id=cf5c9c7caaf4451a9b1ca1a1095408d1&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=dc7e827817764fa2ba7792621216b8d6 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:52.024969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a8dd-0587-4345-8999-da011bd174df None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.025556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a8dd-0587-4345-8999-da011bd174df None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.025556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a8dd-0587-4345-8999-da011bd174df None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.027066 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 523/1046] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:52.032107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.032585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.032685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.051227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:52.052433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:52.069097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091793db-a17b-42ee-ba91-e67b1059d574 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.069386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091793db-a17b-42ee-ba91-e67b1059d574 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.069667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091793db-a17b-42ee-ba91-e67b1059d574 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.071181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 524/1047] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:52.076212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.076507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.076776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.098278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:52.099488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:52.121652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:52.129953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 524/1048] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:52.171800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 525/1049] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:52.411040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:52.462237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:52.516470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:52.520432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40273-5b80-474c-b2d3-3bc4451d29b2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:52.559543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:52.560285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.560574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.560852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] PATH_INFO: `/v3/services/image` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.561440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:52.562841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:52.573265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:52.574364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:44:52.579029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:52.579772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f4cf-8edd-4944-ac8c-136750d218b6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:52.579849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 525/1050] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 1 13:44:52 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:52.583906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:52.612857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:52.613484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.613764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.614072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.625248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:52.630795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:52.631979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:52.632844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 526/1051] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:52.637934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:52.657157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:52.715853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84227afb-bfd7-4e71-b997-17214c656363 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:52.751289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeed34c-0c82-4223-935e-46ab5e1abd55 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:52.754355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:52.755105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.755459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.755741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.756382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] RBAC: Adding query filter params (name=image) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:52.756690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:52.758073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:52.763729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 526/1052] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:52.769346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:52.792807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:52.793474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.793751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.794065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.794728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:52.795029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:52.796393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ffc97-53e9-4b00-9e2a-413f8bf5ab42 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:52.801239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:52.805885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 527/1053] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:52.807227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879439a0-5b07-4086-92c0-bbf011e34fad None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.807523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879439a0-5b07-4086-92c0-bbf011e34fad None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.807807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879439a0-5b07-4086-92c0-bbf011e34fad None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.809021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 528/1054] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:52.811748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879439a0-5b07-4086-92c0-bbf011e34fad None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:52.873999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c023f9-7631-4a43-89f0-279f6fb477cb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:52.893148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:52.911017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:52.911735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.912020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.912293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.912932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] RBAC: Adding query filter params (type=image) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:52.913238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:52.914620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df336-ea13-42b7-a592-4688e301d409 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:52.917823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879439a0-5b07-4086-92c0-bbf011e34fad None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:52.919635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 527/1055] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:52.920826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.921208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.921488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.940254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:52.941434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:52.954387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0543-781e-496e-992f-43080de73a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:52.955076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0543-781e-496e-992f-43080de73a1b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:52.955510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0543-781e-496e-992f-43080de73a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:52.955899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0543-781e-496e-992f-43080de73a1b None admin] PATH_INFO: `/v3/users/7e73e98f6c6f4682b81ec5c131d3b9c1` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:52.969677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0543-781e-496e-992f-43080de73a1b None admin] RBAC: Authorizing `identity:get_user(user_id=7e73e98f6c6f4682b81ec5c131d3b9c1)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:52.971694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0543-781e-496e-992f-43080de73a1b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:52.973334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 529/1056] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:44:52 2022] GET /identity/v3/users/7e73e98f6c6f4682b81ec5c131d3b9c1 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:52.974120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0543-781e-496e-992f-43080de73a1b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.012235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 528/1057] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:53.013168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.043552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:53.091968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0543-781e-496e-992f-43080de73a1b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.133035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.133818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.134139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed83bc-afc6-4a97-ad64-c883801850ae None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.134325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.134721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.135518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:53.137429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:53.144457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:53.174028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.174743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.175027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.175297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.248600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:53.252615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:53.275053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:53.295041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 530/1058] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:44:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:44:53.300260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.383996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 529/1059] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:53.389410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.435550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2bb33-ae60-40bb-b2b4-5f6642f50b8c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.478686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.479510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.479927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.480207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.495018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:53.499438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:53.500481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:53.501352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 531/1060] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:53.507580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.523641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb9dc-a363-462a-85f9-3a4a70fe1817 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.551340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:53.562980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.563617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.563899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.564454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.566213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] RBAC: Authorizing `identity:get_project(project_id=a54e082c19fd45d8bf3c6e067d92478e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:53.567725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:53.569410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 530/1061] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:53.576710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.577578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:53.638423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f28186-5fdd-4413-8e29-15d37b37a4d3 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.681825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.682472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.682755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.683032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.683695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:53.684087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:53.685383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:53.703094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 532/1062] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:53.708050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.711315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ab12-4d22-4537-a651-8c3fc15cd859 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.753559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.754471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.754999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.755457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.756346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=a54e082c19fd45d8bf3c6e067d92478e, user.id=7e73e98f6c6f4682b81ec5c131d3b9c1) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:53.756812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:53.758440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:53.764736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 531/1063] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/role_assignments?user.id=7e73e98f6c6f4682b81ec5c131d3b9c1&scope.project.id=a54e082c19fd45d8bf3c6e067d92478e&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:53.830831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89bbf-a4a0-43f8-a910-9ac22ef5f964 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:53.843902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:53.873027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:53.874168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:53.874709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:53.875373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:53.891639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:53.893208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:53.911953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:53.913186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:53.914345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:53.914345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:53.914345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:53.914345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 533/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:53.919084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:53.985764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:54.044116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a409718-3805-4751-8f3f-4049c7ee4377 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:54.083861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f593d-3d79-429f-9801-529d0379600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:54.084487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f593d-3d79-429f-9801-529d0379600e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:54.084765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f593d-3d79-429f-9801-529d0379600e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:54.085048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f593d-3d79-429f-9801-529d0379600e None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:54.085722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f593d-3d79-429f-9801-529d0379600e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:54.086054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f593d-3d79-429f-9801-529d0379600e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:54.087434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f593d-3d79-429f-9801-529d0379600e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:54.101278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 532/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:54.106751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:54.224147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff872ed-690c-4a30-978b-98c6b479735d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:54.262012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:54.262649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:54.262926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:54.263197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:54.272753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:54.274168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:54.287094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:54.288073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:54.289022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 534/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:44:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:54.295224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f593d-3d79-429f-9801-529d0379600e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:54.426639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f593d-3d79-429f-9801-529d0379600e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:54.446124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:54.475741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:54.476358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:54.476644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:54.476916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:54.477658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:54.477986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:54.479407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:54.510719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 533/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:44:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:54.512725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:54.543491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:54.645461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a46b-cf58-43ed-b0ce-eeba1fe963e7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:54.685146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:54.685782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:54.686085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:54.686359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:54.700056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:54.701508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:54.716755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:54.722188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:54.723239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:54.723239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:54.723239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:54.723239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 535/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:54.726472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:54.842439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9fa6a-6443-4fad-90c6-fe0b560224b9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:54.885606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:54.887046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:54.887335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:54.887615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:54.888274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:54.888580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:54.889991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:54.907927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 534/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:54.909790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:54.995174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:55.020431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20421ec-71e6-4253-aeed-5e6f181dcea9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:55.057763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:55.058423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.058699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.058967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.066806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:55.068247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:55.079695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:55.080710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:55.081612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 536/1070] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:55.088087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:55.123620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:55.200359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9fb57-7c9f-44e9-b69b-60fd09004951 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:55.237129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:55.237943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.238328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.238707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.239504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:55.239916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:55.241700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f883af-b8fd-4033-ae0d-9c02f29874ea None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:55.253935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 535/1071] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:55.259988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:55.367834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7ee7-d660-4601-96b1-68e9e743cb18 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:55.404343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7744-f845-41a1-8792-ac71225cb049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:55.405178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7744-f845-41a1-8792-ac71225cb049 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.405533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7744-f845-41a1-8792-ac71225cb049 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.405918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7744-f845-41a1-8792-ac71225cb049 None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/cf5c9c7caaf4451a9b1ca1a1095408d1/roles/dc7e827817764fa2ba7792621216b8d6` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.408187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7744-f845-41a1-8792-ac71225cb049 None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=cf5c9c7caaf4451a9b1ca1a1095408d1, role_id=dc7e827817764fa2ba7792621216b8d6)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:55.410403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7744-f845-41a1-8792-ac71225cb049 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:55.412891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:55.454724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caea18e-91d5-44dd-b434-9f06c53566ee None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.454950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caea18e-91d5-44dd-b434-9f06c53566ee None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.455766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caea18e-91d5-44dd-b434-9f06c53566ee None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.457204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 536/1072] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:55.462932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fea7b-02b8-4773-afd9-08ff62e77523 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.463221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fea7b-02b8-4773-afd9-08ff62e77523 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.463495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fea7b-02b8-4773-afd9-08ff62e77523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.478291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 537/1073] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:44:55 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/cf5c9c7caaf4451a9b1ca1a1095408d1/roles/dc7e827817764fa2ba7792621216b8d6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:44:55.489486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4d3fea7b-02b8-4773-afd9-08ff62e77523 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:55.490787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4d3fea7b-02b8-4773-afd9-08ff62e77523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:55.567535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 537/1074] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:55.825002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18194354-bf3c-4ea8-8da3-663f80dcc65a None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.825387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18194354-bf3c-4ea8-8da3-663f80dcc65a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.825594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18194354-bf3c-4ea8-8da3-663f80dcc65a None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.827103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 538/1075] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:44:55.832343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:55.832691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:55.832910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:55.844823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18194354-bf3c-4ea8-8da3-663f80dcc65a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:55.857561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:55.858813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:55.881639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:55.940322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 538/1076] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:55.962384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18194354-bf3c-4ea8-8da3-663f80dcc65a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:55.999440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.000058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.000336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.000609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.064666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:56.072084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:56.073601 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:56.178022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 539/1077] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:44:56.183372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:56.210465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:56.295029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bdbd2-37bb-4d00-95ef-298cb1f163e9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:56.310742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:56.318569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2728d-e6e6-45b9-9f58-248965e8d6ef None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:56.334996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.335673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.335996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.336292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.348251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:56.349697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:56.356857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.357586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.357865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.358168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:56.366835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:56.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:56.368914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:56.368914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:56.368914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:56.368914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 539/1078] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:56.372919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:56.373758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:56.377845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:56.378013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:56.379071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:56.379951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 540/1079] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:56.384821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:56.485961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e195c3b-4608-487a-b788-df300e6989d1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:56.494680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cbd43-a4ff-46d5-8768-3831d3ea2147 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:56.523536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.524154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.524437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.524711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.525368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:56.525687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:56.527076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:56.534462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.535080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.535364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.535643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.536311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:56.536622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:56.538033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:56.540437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 540/1080] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:56.549571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 541/1081] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:56.555454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:56.639356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:56.682770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07782a05-5061-476f-b077-a56f1f7f6472 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:56.720423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.721132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.721639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.722020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] PATH_INFO: `/v3/users/7e73e98f6c6f4682b81ec5c131d3b9c1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.723240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] RBAC: Authorizing `identity:get_user(user_id=7e73e98f6c6f4682b81ec5c131d3b9c1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:56.725629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:56.726677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 541/1082] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:44:56 2022] GET /identity/v3/users/7e73e98f6c6f4682b81ec5c131d3b9c1 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:56.732518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:56.758926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:56.852196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490859d-de38-47f9-948a-f2bf81f7f775 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:56.889157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6690179-1c30-4313-aba9-4f9862551de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:56.889795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6690179-1c30-4313-aba9-4f9862551de1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:56.890122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6690179-1c30-4313-aba9-4f9862551de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:56.890398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6690179-1c30-4313-aba9-4f9862551de1 None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:56.891576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6690179-1c30-4313-aba9-4f9862551de1 None admin] RBAC: Authorizing `identity:get_project(project_id=a54e082c19fd45d8bf3c6e067d92478e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:56.893618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6690179-1c30-4313-aba9-4f9862551de1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:56.895327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 542/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:44:56 2022] GET /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:56.901400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:57.012849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97369c25-df54-43d1-b640-003da1f303c4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:57.043936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:57.049001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b891db-5112-4c9e-a079-d899e67db9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:57.049759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b891db-5112-4c9e-a079-d899e67db9d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:57.050078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b891db-5112-4c9e-a079-d899e67db9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:57.050356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b891db-5112-4c9e-a079-d899e67db9d9 None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e/users/7e73e98f6c6f4682b81ec5c131d3b9c1/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:57.052250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b891db-5112-4c9e-a079-d899e67db9d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a54e082c19fd45d8bf3c6e067d92478e, user_id=7e73e98f6c6f4682b81ec5c131d3b9c1, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:57.054033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b891db-5112-4c9e-a079-d899e67db9d9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:57.156329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 542/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:44:56 2022] PUT /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e/users/7e73e98f6c6f4682b81ec5c131d3b9c1/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:44:57.431435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf8dc1-5a39-4d56-a9a9-cb8495bbdb24 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:57.431962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf8dc1-5a39-4d56-a9a9-cb8495bbdb24 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:57.431962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf8dc1-5a39-4d56-a9a9-cb8495bbdb24 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:57.433247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 543/1085] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:57.438568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:57.438999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:57.439093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:57.459860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:57.461316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:57.532558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 543/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:44:57.803113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf8dc1-5a39-4d56-a9a9-cb8495bbdb24 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:57.899564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:57.916497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf8dc1-5a39-4d56-a9a9-cb8495bbdb24 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:57.950687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:57.951316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:57.951594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:57.951865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:57.962120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:57.966879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:57.967792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:57.969050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 544/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 13:44:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:57.974826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:58.073575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2cd8-b4a4-4c84-9896-5ca0b394f0f0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:58.096056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:58.108233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:58.108843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:58.109127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:58.109492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:58.110137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:58.110442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:58.111909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:58.120636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 544/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:44:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:58.126412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:58.236861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7828ceb-e482-4617-8b32-a86442dfac4d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:58.272570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:58.273204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:58.273484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:58.273792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:58.284862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:58.286320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:58.289039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:58.301812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:58.307737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:58.308910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:58.308910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:58.308910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:58.308910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 545/1089] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:58.310564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:58.412308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdbd1e-fb27-48a0-a197-fbd317ee80f7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:58.447171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:58.447781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:58.447781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:58.448061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:58.448560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:58.448862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:58.450268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:58.462214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 545/1090] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:58.467994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:58.569697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c15c29-ebab-4beb-93a3-07aa8c29c1b2 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:58.606401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:58.607034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:58.607314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:58.607591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:58.615259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:58.616967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:44:58.628135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:58.629063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:58.630034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 546/1091] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 13:44:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:58.635310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:58.682094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:58.735771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ce16-b87e-4208-bb73-b91c45cf762f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:58.770356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:58.770964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:58.771269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:58.771541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:58.772263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:58.772566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:58.773989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd0d4-7856-4947-83db-02069aae2424 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:58.790845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 546/1092] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 13:44:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:58.796901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:58.854758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:58.903299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27059d08-7c2c-4fbb-8b24-ac8ce69fa899 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:58.940141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:58.940871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:58.941154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:58.941430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.019341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.020802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.064865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95752f46-a109-4a4b-9ea9-139ae73667a8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.065359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95752f46-a109-4a4b-9ea9-139ae73667a8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.065789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95752f46-a109-4a4b-9ea9-139ae73667a8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.067257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 547/1093] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:44:59.072689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.072979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.073260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.085511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.091598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:44:59.092822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:44:59.134749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 547/1094] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:44:59.140934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.165533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 548/1095] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:44:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:44:59.257096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f7853-d031-4221-abe0-c76963de8438 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.297765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.298428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.299382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.299658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.310141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.311582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:44:59.327656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:59.328671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:44:59.329553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:44:59.329553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:44:59.329553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:59.329553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 548/1096] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:59.335238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.365680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.438715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.440768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11476454-07fb-41e2-ac52-9cfedb8c9fd5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.451905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.482571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.483205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.483458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.483736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.484380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:59.484685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.486163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.497852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 549/1097] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:44:59.504498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.554248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cf794-9b7e-47ab-b925-331d6bf09f4b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.594095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.594856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.595152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.595561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.609326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.610314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:44:59.615250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:59.616449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:44:59.617528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 549/1098] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:44:59.618525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149de98-7ca6-411f-9d03-e9d83d1cdca8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.623290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.655245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.655888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.656171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.656453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.664388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.666255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:44:59.678287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:44:59.679336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:44:59.680353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 550/1099] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:44:59.685512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.697130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.733446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db48f8-7da8-4e71-b31d-6ea4a61b100b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.770598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.771302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.771686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.772030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.772705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:59.773011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.774417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.783109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 550/1100] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:59.789213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.795014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db2e2e-81d9-44ec-9794-e7f349c1dd88 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.832032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.832570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.832849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.833122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.833811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:44:59.834151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.835566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.848177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 551/1101] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:59.856517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.872870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.909357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ae8c8-a961-4106-bc48-fcde3e868bd3 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:44:59.940856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:44:59.948136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:44:59.948780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:44:59.949123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:44:59.949402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] PATH_INFO: `/v3/users/7e73e98f6c6f4682b81ec5c131d3b9c1` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:44:59.950569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] RBAC: Authorizing `identity:get_user(user_id=7e73e98f6c6f4682b81ec5c131d3b9c1)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:44:59.952040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:44:59.953632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 551/1102] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/users/7e73e98f6c6f4682b81ec5c131d3b9c1 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:44:59.959415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:44:59.980082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c704c02-c314-4a09-aa2d-a149b8f22499 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:00.017981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:00.018591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:00.018875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:00.019148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:00.019875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] RBAC: Adding query filter params (role.id=dc7e827817764fa2ba7792621216b8d6, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=cf5c9c7caaf4451a9b1ca1a1095408d1) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:00.020184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:00.021599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:00.027522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 552/1103] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/role_assignments?user.id=cf5c9c7caaf4451a9b1ca1a1095408d1&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=dc7e827817764fa2ba7792621216b8d6 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:45:00.072000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60e67-9c3a-4206-840a-6f19cc5d0548 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:00.109303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:00.109836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:00.110143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:00.110416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:00.181689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:00.188439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:00.190094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:00.295401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 552/1104] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:44:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:00.302072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:00.385055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:00.409788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57162e31-39a9-4654-a77d-c49e6cc62d03 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:00.445315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90669c0-437a-40cb-b483-5c91adca317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:00.445992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90669c0-437a-40cb-b483-5c91adca317b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:00.446275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90669c0-437a-40cb-b483-5c91adca317b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:00.446552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90669c0-437a-40cb-b483-5c91adca317b None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:00.447694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90669c0-437a-40cb-b483-5c91adca317b None admin] RBAC: Authorizing `identity:get_project(project_id=a54e082c19fd45d8bf3c6e067d92478e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:00.449124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90669c0-437a-40cb-b483-5c91adca317b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:00.451115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 553/1105] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:00 2022] GET /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:00.457228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:00.560429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:00.564705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1a40f-2155-4ba0-9205-b5f889978b11 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:00.602729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:00.603656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:00.603771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:00.604015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:00.604756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=a54e082c19fd45d8bf3c6e067d92478e, user.id=7e73e98f6c6f4682b81ec5c131d3b9c1) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:00.605065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:00.606604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2184bc8-3a51-42ed-8233-57796eca5899 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:00.614187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 553/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:00 2022] GET /identity/v3/role_assignments?user.id=7e73e98f6c6f4682b81ec5c131d3b9c1&scope.project.id=a54e082c19fd45d8bf3c6e067d92478e&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:02.524219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dba22-a2a6-4b89-8b55-2505be50cc75 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:02.524863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dba22-a2a6-4b89-8b55-2505be50cc75 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:02.524863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dba22-a2a6-4b89-8b55-2505be50cc75 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:02.526234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 554/1107] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:02.531402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:02.531835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:02.531948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:02.550622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:02.551821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:02.619269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 554/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:02.886483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dba22-a2a6-4b89-8b55-2505be50cc75 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:02.988543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dba22-a2a6-4b89-8b55-2505be50cc75 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:03.022321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e94be-6c2a-4bd7-918e-6d7ef8fde61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:03.022942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e94be-6c2a-4bd7-918e-6d7ef8fde61c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:03.023222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e94be-6c2a-4bd7-918e-6d7ef8fde61c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:03.023498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e94be-6c2a-4bd7-918e-6d7ef8fde61c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:03.024632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e94be-6c2a-4bd7-918e-6d7ef8fde61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:03.026040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e94be-6c2a-4bd7-918e-6d7ef8fde61c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:03.027590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 555/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:03.034369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:03.124306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:03.132977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff80df-0236-41c6-9992-0b3d83d30e93 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:03.168583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c195a-10ae-4016-89b0-62493b3d2e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:03.169131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c195a-10ae-4016-89b0-62493b3d2e48 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:03.169410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c195a-10ae-4016-89b0-62493b3d2e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:03.169681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c195a-10ae-4016-89b0-62493b3d2e48 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:03.170363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c195a-10ae-4016-89b0-62493b3d2e48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:03.171796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c195a-10ae-4016-89b0-62493b3d2e48 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:03.263988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 555/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:45:03 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:05.185304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82799dc-aa66-4c37-a989-b33a81d14fb4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:05.185878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82799dc-aa66-4c37-a989-b33a81d14fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:05.185878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82799dc-aa66-4c37-a989-b33a81d14fb4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:05.187227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 556/1111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:05.192717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:05.193284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:05.193284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:05.211749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:05.212942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:05.279714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 556/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:05.599883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82799dc-aa66-4c37-a989-b33a81d14fb4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:05.599883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:05.656094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82799dc-aa66-4c37-a989-b33a81d14fb4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:05.688540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:05.689791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:05.690108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:05.690388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] PATH_INFO: `/v3/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:05.691780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] RBAC: Authorizing `identity:get_role(role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:05.693128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:05.694514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 557/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:05 2022] GET /identity/v3/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:05.700428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:05.779376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:05.800752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431942c-2d53-4253-96fb-46d9946e1cb1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:05.859519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:05.860084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:05.860367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:05.861332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] PATH_INFO: `/v3/users/4bfdb968fd6b4c20a9050f71c50c158b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:05.875082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] RBAC: Authorizing `identity:get_user(user_id=4bfdb968fd6b4c20a9050f71c50c158b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:05.879790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:05.879790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 557/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:05 2022] GET /identity/v3/users/4bfdb968fd6b4c20a9050f71c50c158b => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:05.889008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:05.969517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:05.988586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823b04a-2430-4232-9411-724fb2ef36df None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:06.022501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:06.023104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:06.023386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:06.023676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:06.088661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:06.090121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:06.167110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:06.186643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 558/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:06.192808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:06.293412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee043e0f-72ee-4db6-976b-8427b6dfcba7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:06.326744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b582e-3d86-47e9-9b2d-23944d083c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:06.327366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b582e-3d86-47e9-9b2d-23944d083c6a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:06.327648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b582e-3d86-47e9-9b2d-23944d083c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:06.327923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b582e-3d86-47e9-9b2d-23944d083c6a None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:06.329034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b582e-3d86-47e9-9b2d-23944d083c6a None admin] RBAC: Authorizing `identity:get_project(project_id=a54e082c19fd45d8bf3c6e067d92478e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:06.330645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b582e-3d86-47e9-9b2d-23944d083c6a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:06.332268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 558/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:06 2022] GET /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:06.338787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:06.437081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a526fd-6a2f-4af3-b4e3-02b7fb4ab051 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:06.469527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:06.470722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:06.471000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:06.471274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:06.471998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] RBAC: Adding query filter params (role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=a54e082c19fd45d8bf3c6e067d92478e, user.id=4bfdb968fd6b4c20a9050f71c50c158b) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:06.472309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:06.473731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cea002-19db-47b4-947e-b5dc2d46a17f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:06.478975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 559/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:06 2022] GET /identity/v3/role_assignments?user.id=4bfdb968fd6b4c20a9050f71c50c158b&scope.project.id=a54e082c19fd45d8bf3c6e067d92478e&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:08.351637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489668-fd98-4145-b64e-74d3fa033d08 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:08.352213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489668-fd98-4145-b64e-74d3fa033d08 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:08.352213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489668-fd98-4145-b64e-74d3fa033d08 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:08.353510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 559/1118] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:08.358654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:08.359074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:08.359208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:08.379636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:08.380837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:08.420970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:08.446732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 560/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:08.719303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489668-fd98-4145-b64e-74d3fa033d08 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:08.730004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:08.823190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489668-fd98-4145-b64e-74d3fa033d08 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:08.856854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:08.857487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:08.857770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:08.858080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] PATH_INFO: `/v3/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:08.859491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] RBAC: Authorizing `identity:get_role(role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:08.861206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:08.862598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 560/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:08 2022] GET /identity/v3/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:08.868744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:08.966765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780eded-4bae-4d3f-804c-949b3409e919 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.002472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.003104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.003391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.003672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] PATH_INFO: `/v3/users/4bfdb968fd6b4c20a9050f71c50c158b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.004797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] RBAC: Authorizing `identity:get_user(user_id=4bfdb968fd6b4c20a9050f71c50c158b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.006280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.007861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 561/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:08 2022] GET /identity/v3/users/4bfdb968fd6b4c20a9050f71c50c158b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:09.013634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.082449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:09.111061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22677b-85fc-45bd-b11b-3ca791cebec1 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.145122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83a59a-40ad-46b3-b89d-c33a1035a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.145752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83a59a-40ad-46b3-b89d-c33a1035a07e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.146059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83a59a-40ad-46b3-b89d-c33a1035a07e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.146331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83a59a-40ad-46b3-b89d-c33a1035a07e None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.147446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83a59a-40ad-46b3-b89d-c33a1035a07e None admin] RBAC: Authorizing `identity:get_project(project_id=a54e082c19fd45d8bf3c6e067d92478e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.148926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83a59a-40ad-46b3-b89d-c33a1035a07e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.150582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 561/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:09.156598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.204209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:09.260757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456599c-8e3b-44e6-9b02-5c6e97bae370 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.293758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d993b-6c0f-41a6-86ec-fac4bec6428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.294439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d993b-6c0f-41a6-86ec-fac4bec6428f None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.294723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d993b-6c0f-41a6-86ec-fac4bec6428f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.295000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d993b-6c0f-41a6-86ec-fac4bec6428f None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e/users/4bfdb968fd6b4c20a9050f71c50c158b/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.297094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d993b-6c0f-41a6-86ec-fac4bec6428f None admin] RBAC: Authorizing `identity:create_grant(project_id=a54e082c19fd45d8bf3c6e067d92478e, user_id=4bfdb968fd6b4c20a9050f71c50c158b, role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.299171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d993b-6c0f-41a6-86ec-fac4bec6428f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.425122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 562/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:45:09 2022] PUT /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e/users/4bfdb968fd6b4c20a9050f71c50c158b/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:45:11.337540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384412-30d4-453b-8b7d-98d707b950e3 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:11.338138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384412-30d4-453b-8b7d-98d707b950e3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:11.338138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384412-30d4-453b-8b7d-98d707b950e3 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:11.339473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 562/1124] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:11.345343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:11.345766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:11.345889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:11.363596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:11.364777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:11.430315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 563/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:11.701170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384412-30d4-453b-8b7d-98d707b950e3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:11.801612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384412-30d4-453b-8b7d-98d707b950e3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:11.835058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:11.835681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:11.835962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:11.836244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] PATH_INFO: `/v3/roles/d13c72fdd3c54d049df3e12a145205e8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:11.837625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] RBAC: Authorizing `identity:get_role(role_id=d13c72fdd3c54d049df3e12a145205e8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:11.839025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:11.840334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 563/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:11 2022] GET /identity/v3/roles/d13c72fdd3c54d049df3e12a145205e8 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:11.846429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:11.877013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:11.943042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58acac-ea87-4563-b59a-8328e17524b9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:11.975406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:11.976026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:11.976305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:11.976577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] PATH_INFO: `/v3/users/4bfdb968fd6b4c20a9050f71c50c158b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:11.977675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] RBAC: Authorizing `identity:get_user(user_id=4bfdb968fd6b4c20a9050f71c50c158b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:11.979124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:11.980694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 564/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:11 2022] GET /identity/v3/users/4bfdb968fd6b4c20a9050f71c50c158b => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:11.987200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:11.998226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:12.087749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec520489-3368-4a00-9db0-205daca185ef None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:12.121407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:12.122059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:12.122345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:12.122622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:12.194429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:12.195823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:12.205627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:12.294350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 564/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:12.300526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:12.398299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:12.401213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b00589-8078-41c8-a496-50175d0ae60d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:12.435138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36004b0-1e3e-4a67-9d9a-310113242030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:12.435716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36004b0-1e3e-4a67-9d9a-310113242030 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:12.435999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36004b0-1e3e-4a67-9d9a-310113242030 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:12.436276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36004b0-1e3e-4a67-9d9a-310113242030 None admin] PATH_INFO: `/v3/projects/a54e082c19fd45d8bf3c6e067d92478e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:12.437384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36004b0-1e3e-4a67-9d9a-310113242030 None admin] RBAC: Authorizing `identity:get_project(project_id=a54e082c19fd45d8bf3c6e067d92478e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:12.439599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36004b0-1e3e-4a67-9d9a-310113242030 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:12.441222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 565/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:12 2022] GET /identity/v3/projects/a54e082c19fd45d8bf3c6e067d92478e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:12.447510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:12.544642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c28599-caf5-4f4d-854b-ec59975ce659 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:12.554422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:12.579804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86610b-6b92-410f-a772-683924612831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:12.580404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86610b-6b92-410f-a772-683924612831 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:12.580682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86610b-6b92-410f-a772-683924612831 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:12.580953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86610b-6b92-410f-a772-683924612831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:12.581679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86610b-6b92-410f-a772-683924612831 None admin] RBAC: Adding query filter params (role.id=d13c72fdd3c54d049df3e12a145205e8, scope.project.id=a54e082c19fd45d8bf3c6e067d92478e, user.id=4bfdb968fd6b4c20a9050f71c50c158b) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:12.582020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86610b-6b92-410f-a772-683924612831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:12.583449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86610b-6b92-410f-a772-683924612831 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:12.589826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 565/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:12 2022] GET /identity/v3/role_assignments?user.id=4bfdb968fd6b4c20a9050f71c50c158b&scope.project.id=a54e082c19fd45d8bf3c6e067d92478e&role.id=d13c72fdd3c54d049df3e12a145205e8 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:14.516983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1617f08-8e81-41bd-a7f5-db7916592c4d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.517556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1617f08-8e81-41bd-a7f5-db7916592c4d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.517556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1617f08-8e81-41bd-a7f5-db7916592c4d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.518911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 566/1131] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:14.524036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.524467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.524574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.543075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:14.544296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:14.612612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 566/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:14.895983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1617f08-8e81-41bd-a7f5-db7916592c4d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.994058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1617f08-8e81-41bd-a7f5-db7916592c4d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:15.027328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e32185-b8e3-4a42-88ca-6b361cd0c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:15.027946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e32185-b8e3-4a42-88ca-6b361cd0c604 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:15.028225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e32185-b8e3-4a42-88ca-6b361cd0c604 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:15.028498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e32185-b8e3-4a42-88ca-6b361cd0c604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:15.029587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e32185-b8e3-4a42-88ca-6b361cd0c604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:15.031021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e32185-b8e3-4a42-88ca-6b361cd0c604 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:15.032726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 567/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:15.038894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:15.137615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db35f6-cc56-4265-bd86-06788c9c8439 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:15.139169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:15.173224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22d96d-e104-4c9c-abcb-e5ba1c7a1af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:15.173791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22d96d-e104-4c9c-abcb-e5ba1c7a1af4 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:15.174099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22d96d-e104-4c9c-abcb-e5ba1c7a1af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:15.174380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22d96d-e104-4c9c-abcb-e5ba1c7a1af4 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:15.175036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22d96d-e104-4c9c-abcb-e5ba1c7a1af4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:15.176469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22d96d-e104-4c9c-abcb-e5ba1c7a1af4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:15.261664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 567/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:45:15 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:17.218810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef344d9b-c4ba-42bc-b7b0-b58b590ec406 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.219387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef344d9b-c4ba-42bc-b7b0-b58b590ec406 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.219387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef344d9b-c4ba-42bc-b7b0-b58b590ec406 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.220745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 568/1135] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:17.226231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.226715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.226816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.245776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:17.247001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:17.313875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 568/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:17.582973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef344d9b-c4ba-42bc-b7b0-b58b590ec406 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:17.620430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:17.684287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef344d9b-c4ba-42bc-b7b0-b58b590ec406 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:17.716581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e93a19-4481-4dd0-a9b4-753bf270d4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:17.717200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e93a19-4481-4dd0-a9b4-753bf270d4bf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.717480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e93a19-4481-4dd0-a9b4-753bf270d4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.717754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e93a19-4481-4dd0-a9b4-753bf270d4bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.719469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e93a19-4481-4dd0-a9b4-753bf270d4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:17.720890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e93a19-4481-4dd0-a9b4-753bf270d4bf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:17.722532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 569/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:17.728486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:17.811561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:17.826652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548e2a0-60d0-4d41-b3ac-27d782ec449f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:17.861017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dd96e-a026-4cbd-8b49-f444652743c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:17.861616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dd96e-a026-4cbd-8b49-f444652743c6 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.861925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dd96e-a026-4cbd-8b49-f444652743c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.862209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dd96e-a026-4cbd-8b49-f444652743c6 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.862885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dd96e-a026-4cbd-8b49-f444652743c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:17.864324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dd96e-a026-4cbd-8b49-f444652743c6 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:17.960690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 569/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:45:17 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:19.936327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abff4-610b-48ff-bcae-c0ba5247882b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.936856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abff4-610b-48ff-bcae-c0ba5247882b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.936856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abff4-610b-48ff-bcae-c0ba5247882b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.938811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 570/1139] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:19.944059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.944495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.944617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.963350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:19.964538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:20.034178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 570/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:20.302745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9abff4-610b-48ff-bcae-c0ba5247882b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:20.406112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9abff4-610b-48ff-bcae-c0ba5247882b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:20.409095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:20.441365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:20.442074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:20.442597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:20.443046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:20.454258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:20.459081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:20.460248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:45:20.461533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 571/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:20.466330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:20.534508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:20.564576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0fa55-fb7b-44c7-9fcd-ec165de83d28 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:20.599130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:20.599749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:20.600409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:20.600409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:20.600945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:20.601248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:20.602703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:20.611832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 571/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:20.617858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:20.716087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d62df8-8401-458b-a4aa-6a1605c827e4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:20.777099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:20.777722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:20.778035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:20.778308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] PATH_INFO: `/v3/users/a216c0dcf43e4d8b8e8ca176997653bd` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:20.791265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] RBAC: Authorizing `identity:get_user(user_id=a216c0dcf43e4d8b8e8ca176997653bd)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:20.792669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:20.794288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 572/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:20 2022] GET /identity/v3/users/a216c0dcf43e4d8b8e8ca176997653bd => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:20.801027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:20.902209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2cf09-3dba-40b1-a127-a7e542bb841f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:20.931516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:20.936341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:20.936952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:20.937236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:20.938183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:21.005619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:21.007104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:21.101136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 572/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:21.107161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:21.132748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:21.208827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0224c-ad85-4d48-b158-3a610d562805 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:21.243646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b6a0-f2e6-4ce6-8a37-8a197ec37c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:21.244273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b6a0-f2e6-4ce6-8a37-8a197ec37c5d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:21.244555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b6a0-f2e6-4ce6-8a37-8a197ec37c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:21.244838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b6a0-f2e6-4ce6-8a37-8a197ec37c5d None admin] PATH_INFO: `/v3/projects/96c93b7243ed4348a5a3c556d97bfe57` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:21.245981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0b6a0-f2e6-4ce6-8a37-8a197ec37c5d None admin] RBAC: Authorizing `identity:get_project(project_id=96c93b7243ed4348a5a3c556d97bfe57)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:21.247410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0b6a0-f2e6-4ce6-8a37-8a197ec37c5d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:21.249026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 573/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:21 2022] GET /identity/v3/projects/96c93b7243ed4348a5a3c556d97bfe57 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:21.255339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:21.285556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:21.353842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566a26-daf4-40dd-a71a-96a2369de88a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:21.387920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:21.388525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:21.388802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:21.389079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:21.389799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=96c93b7243ed4348a5a3c556d97bfe57, user.id=a216c0dcf43e4d8b8e8ca176997653bd) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:21.390146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:21.391785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18583-9179-4119-a5c5-f85a3fe0f7e7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:21.397247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 573/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:21 2022] GET /identity/v3/role_assignments?user.id=a216c0dcf43e4d8b8e8ca176997653bd&scope.project.id=96c93b7243ed4348a5a3c556d97bfe57&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:23.279561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8becb-c2a9-4675-b89b-9601cd2aa62c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.280104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8becb-c2a9-4675-b89b-9601cd2aa62c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.280104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8becb-c2a9-4675-b89b-9601cd2aa62c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.282042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 574/1147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:23.287256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.287691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.287783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.305743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:23.306950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:23.373090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 574/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:45:23.637070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8becb-c2a9-4675-b89b-9601cd2aa62c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.726593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.737396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8becb-c2a9-4675-b89b-9601cd2aa62c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.772470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.773092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.773377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.773655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.783588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:23.788338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:23.789428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:45:23.790398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 575/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:23.795512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.822719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.895085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6ad58-ca3a-42ce-aa23-1561ab7336aa None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.928307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.928909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.929191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.929466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.930148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:23.930453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:23.931822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:23.940321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 575/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:23.946271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:24.042570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaca862-79e4-451a-81fa-d738b40f5a92 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:24.076151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:24.076769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:24.077044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:24.077318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] PATH_INFO: `/v3/users/a216c0dcf43e4d8b8e8ca176997653bd` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:24.078475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] RBAC: Authorizing `identity:get_user(user_id=a216c0dcf43e4d8b8e8ca176997653bd)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:24.079879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:24.081434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 576/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/users/a216c0dcf43e4d8b8e8ca176997653bd => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:24.087753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:24.176128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:24.187963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991044a-ca34-42e7-99a2-96bd8f02983f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:24.222044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d324a50-585e-4e32-aabc-817196157930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:24.222670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d324a50-585e-4e32-aabc-817196157930 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:24.222951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d324a50-585e-4e32-aabc-817196157930 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:24.223234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d324a50-585e-4e32-aabc-817196157930 None admin] PATH_INFO: `/v3/projects/96c93b7243ed4348a5a3c556d97bfe57` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:24.224344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d324a50-585e-4e32-aabc-817196157930 None admin] RBAC: Authorizing `identity:get_project(project_id=96c93b7243ed4348a5a3c556d97bfe57)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:24.225778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d324a50-585e-4e32-aabc-817196157930 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:24.227431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 576/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:24 2022] GET /identity/v3/projects/96c93b7243ed4348a5a3c556d97bfe57 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:24.233441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:24.253684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:24.332360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba360f-57cc-4f81-bd27-cd0e6f33ac66 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:24.368097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c09ca8-e93d-4c5d-96a1-991e0607597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:24.368755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09ca8-e93d-4c5d-96a1-991e0607597a None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:24.369038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09ca8-e93d-4c5d-96a1-991e0607597a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:24.369316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09ca8-e93d-4c5d-96a1-991e0607597a None admin] PATH_INFO: `/v3/projects/96c93b7243ed4348a5a3c556d97bfe57/users/a216c0dcf43e4d8b8e8ca176997653bd/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:24.371220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c09ca8-e93d-4c5d-96a1-991e0607597a None admin] RBAC: Authorizing `identity:create_grant(project_id=96c93b7243ed4348a5a3c556d97bfe57, user_id=a216c0dcf43e4d8b8e8ca176997653bd, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:24.372659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c09ca8-e93d-4c5d-96a1-991e0607597a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:24.472154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 577/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:45:24 2022] PUT /identity/v3/projects/96c93b7243ed4348a5a3c556d97bfe57/users/a216c0dcf43e4d8b8e8ca176997653bd/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:45:26.364889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7414e-512e-4cce-ba03-ee2531214785 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:26.365430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7414e-512e-4cce-ba03-ee2531214785 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:26.365430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7414e-512e-4cce-ba03-ee2531214785 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:26.366784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 577/1154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:26.372139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:26.372537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:26.372707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:26.391606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:26.392808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:26.458345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 578/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:26.727383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7414e-512e-4cce-ba03-ee2531214785 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:26.832019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7414e-512e-4cce-ba03-ee2531214785 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:26.866475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:26.867198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:26.867482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:26.867762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:26.878564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:26.883352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:26.884696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:45:26.885578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 578/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:26.891766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:26.901786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:26.989179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57fe26-fd9c-445b-9beb-983bdeacfabb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.023288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.023888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.024169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.024443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.025102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:27.025407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.026787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.035375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 579/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:27.041229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.060290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.143010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db56027-f57c-44b0-9383-505b3f0ebb2c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.178223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.178847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.179125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.179396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] PATH_INFO: `/v3/users/a216c0dcf43e4d8b8e8ca176997653bd` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.180517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] RBAC: Authorizing `identity:get_user(user_id=a216c0dcf43e4d8b8e8ca176997653bd)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.181973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.183564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 579/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/users/a216c0dcf43e4d8b8e8ca176997653bd => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:27.189769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.257273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.287397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af30b-dd50-4bb2-982a-1920f3f4dcee None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.319555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.320143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.320421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.320693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.386514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.387940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.452044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.482136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 580/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:27.489109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.572958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.589425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e490d7-9cf0-469c-a3b7-e6a246af0e1d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.625605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59ac65-4c7b-4da0-8fd4-1c1e64025915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.626263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ac65-4c7b-4da0-8fd4-1c1e64025915 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.626548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ac65-4c7b-4da0-8fd4-1c1e64025915 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.626825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ac65-4c7b-4da0-8fd4-1c1e64025915 None admin] PATH_INFO: `/v3/projects/96c93b7243ed4348a5a3c556d97bfe57` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.627941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59ac65-4c7b-4da0-8fd4-1c1e64025915 None admin] RBAC: Authorizing `identity:get_project(project_id=96c93b7243ed4348a5a3c556d97bfe57)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.629381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59ac65-4c7b-4da0-8fd4-1c1e64025915 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.631073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 580/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/projects/96c93b7243ed4348a5a3c556d97bfe57 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:45:27.637225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.735355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1608fc-ff45-4e24-8a7a-82d067853c79 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.768791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.769389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.769672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.769984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.770708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=96c93b7243ed4348a5a3c556d97bfe57, user.id=a216c0dcf43e4d8b8e8ca176997653bd) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:27.771022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.772446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066dc3-146e-464c-ad11-260963df7d71 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.778253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 581/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/role_assignments?user.id=a216c0dcf43e4d8b8e8ca176997653bd&scope.project.id=96c93b7243ed4348a5a3c556d97bfe57&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:29.776913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab46d8-073f-437a-b2f2-1a708943014c None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:29.777478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab46d8-073f-437a-b2f2-1a708943014c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:29.777478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab46d8-073f-437a-b2f2-1a708943014c None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:29.778803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 581/1162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:29.784701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:29.785104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:29.785259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:29.803308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:29.804496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:29.831153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:29.870969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 582/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:30.151821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab46d8-073f-437a-b2f2-1a708943014c None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:30.253882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab46d8-073f-437a-b2f2-1a708943014c None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:30.288389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:30.289003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:30.289283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:30.289557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:30.311545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:30.363011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:30.364722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:30.463317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 582/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:30.469501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:30.567568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b89de-69e2-4217-b365-9175ef8e599d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:30.601078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:30.601576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:30.601856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:30.602165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:30.609515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:30.611082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5c200ba5-0903-4ef2-87c3-e7e2b7dae83b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:30.623274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:30.625142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:45:30.626325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 583/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:45:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:45:30.630930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:30.683159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:30.728865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed64e78-7d78-46a4-9413-3121ab8baec8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:30.762905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:30.763503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:30.763782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:30.764057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:30.764705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:30.765115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:30.766530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a24022-8e17-4ec3-bfd5-7fd5c97bc79e None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:30.773536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 583/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 13:45:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:32.677502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7a162-480a-483f-8b9d-b49fb0d0ae99 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.678037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7a162-480a-483f-8b9d-b49fb0d0ae99 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.678147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7a162-480a-483f-8b9d-b49fb0d0ae99 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.679418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 584/1167] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:32.684468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0407a-c7a8-4c60-9f52-abe42ec2577e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.684875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0407a-c7a8-4c60-9f52-abe42ec2577e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.685031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0407a-c7a8-4c60-9f52-abe42ec2577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.704224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-05a0407a-c7a8-4c60-9f52-abe42ec2577e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:32.705416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-05a0407a-c7a8-4c60-9f52-abe42ec2577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:32.771175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 584/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:33.045973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7a162-480a-483f-8b9d-b49fb0d0ae99 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:33.071026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:33.149190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7a162-480a-483f-8b9d-b49fb0d0ae99 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:33.184618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111eab7-ff99-4713-b42e-cee89e28fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:33.185229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111eab7-ff99-4713-b42e-cee89e28fc15 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:33.185513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111eab7-ff99-4713-b42e-cee89e28fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:33.185789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111eab7-ff99-4713-b42e-cee89e28fc15 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:33.187026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111eab7-ff99-4713-b42e-cee89e28fc15 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:33.188474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111eab7-ff99-4713-b42e-cee89e28fc15 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:33.289851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 585/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Sep 1 13:45:33 2022] POST /identity/v3/domains => generated 267 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:35.193626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b7708-a1e7-4fc8-b053-1af0a5f9ed42 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.194239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b7708-a1e7-4fc8-b053-1af0a5f9ed42 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.194239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b7708-a1e7-4fc8-b053-1af0a5f9ed42 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.195554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 585/1170] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:35.200591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.200999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.201141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.219921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:35.221091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:35.285440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 586/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:35.548554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b7708-a1e7-4fc8-b053-1af0a5f9ed42 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:35.596949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:35.651095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b7708-a1e7-4fc8-b053-1af0a5f9ed42 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:35.685417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14188e7-1424-4edd-b611-dc03118d95c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:35.686078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14188e7-1424-4edd-b611-dc03118d95c4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.686475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14188e7-1424-4edd-b611-dc03118d95c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.686762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14188e7-1424-4edd-b611-dc03118d95c4 None admin] PATH_INFO: `/v3/domains/036023aefeeb4488bae65d43ef9a2b20` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.697078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14188e7-1424-4edd-b611-dc03118d95c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=036023aefeeb4488bae65d43ef9a2b20)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:35.698547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14188e7-1424-4edd-b611-dc03118d95c4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:35.700155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 586/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 1 13:45:35 2022] GET /identity/v3/domains/036023aefeeb4488bae65d43ef9a2b20 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:35.706105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:35.721214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:35.804625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fe3aa-29a5-4967-a304-46536a7e0e47 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:35.837396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257f1d5-fa34-4a50-a0f3-6b9776f3b5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:35.838029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257f1d5-fa34-4a50-a0f3-6b9776f3b5b2 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.838315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257f1d5-fa34-4a50-a0f3-6b9776f3b5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.838592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257f1d5-fa34-4a50-a0f3-6b9776f3b5b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.839322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257f1d5-fa34-4a50-a0f3-6b9776f3b5b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:35.840742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257f1d5-fa34-4a50-a0f3-6b9776f3b5b2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:35.937708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 587/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:45:35 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:37.874762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11061bf1-a37b-4aa5-8fda-54ebe9a61682 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:37.875279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11061bf1-a37b-4aa5-8fda-54ebe9a61682 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:37.875279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11061bf1-a37b-4aa5-8fda-54ebe9a61682 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:37.876576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 587/1174] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:37.882559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:37.882993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:37.883119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:37.901174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:37.902393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:37.961151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:37.969146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 588/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:45:38.245191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11061bf1-a37b-4aa5-8fda-54ebe9a61682 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:38.346207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11061bf1-a37b-4aa5-8fda-54ebe9a61682 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:38.353165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:38.382162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c33b4f-07f0-431b-9a0a-cdd4be3f9e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:38.382807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c33b4f-07f0-431b-9a0a-cdd4be3f9e11 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:38.383118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c33b4f-07f0-431b-9a0a-cdd4be3f9e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:38.383397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c33b4f-07f0-431b-9a0a-cdd4be3f9e11 None admin] PATH_INFO: `/v3/domains/036023aefeeb4488bae65d43ef9a2b20` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:38.384525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c33b4f-07f0-431b-9a0a-cdd4be3f9e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=036023aefeeb4488bae65d43ef9a2b20)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:38.386035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c33b4f-07f0-431b-9a0a-cdd4be3f9e11 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:38.387882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 588/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 1 13:45:38 2022] GET /identity/v3/domains/036023aefeeb4488bae65d43ef9a2b20 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:38.393429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:38.490126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0500a-e265-4f5e-85c3-06597f9f54cd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:38.522909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fe593-4dc0-424a-961f-2dbbfca6e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:38.523439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fe593-4dc0-424a-961f-2dbbfca6e13b None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:38.523725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fe593-4dc0-424a-961f-2dbbfca6e13b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:38.524003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fe593-4dc0-424a-961f-2dbbfca6e13b None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:38.524659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fe593-4dc0-424a-961f-2dbbfca6e13b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:38.526274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fe593-4dc0-424a-961f-2dbbfca6e13b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:38.621947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 589/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:45:38 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:45:40.622284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22417-6f59-49fc-9e7b-905f33671ef3 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:40.622796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22417-6f59-49fc-9e7b-905f33671ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:40.622901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22417-6f59-49fc-9e7b-905f33671ef3 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:40.624230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 589/1178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:40.629404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:40.629866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:40.629997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:40.649081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:40.650310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:40.698828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:40.718845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 590/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:45:40.993572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22417-6f59-49fc-9e7b-905f33671ef3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.100125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22417-6f59-49fc-9e7b-905f33671ef3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.134884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.135508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.135790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.136066 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.146257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:41.151070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:41.152254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:45:41.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 590/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:41.158450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.256846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c6e8b-aa85-4088-8534-02c92043d563 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.293864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.294500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.294788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.295655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.296314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:41.296621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.298002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.305865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 591/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:41.311770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.345706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.410230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae8878-7c82-4c12-9b55-32e6403be4e0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.442848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.443465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.443742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.444013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] PATH_INFO: `/v3/users/13580e40aca346468fb4c5e31039e485` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.456813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] RBAC: Authorizing `identity:get_user(user_id=13580e40aca346468fb4c5e31039e485)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.458254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.459816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 591/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/users/13580e40aca346468fb4c5e31039e485 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:41.466026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.511880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.565657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3004d-1fa7-41eb-9c50-d08dbf6197ee None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.600272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.600417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.600634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.600909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.667625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.669064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.712085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.766576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 592/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:41.772463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.860904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.869572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3d125-d3df-4a0c-9590-be820ceef40d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.903145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6afdfc-5a89-4289-8459-0ee9f18560e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.903761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6afdfc-5a89-4289-8459-0ee9f18560e9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.904036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6afdfc-5a89-4289-8459-0ee9f18560e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.904305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6afdfc-5a89-4289-8459-0ee9f18560e9 None admin] PATH_INFO: `/v3/projects/e7dc12748a714c68a2272505e6743bf9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.906173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6afdfc-5a89-4289-8459-0ee9f18560e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc12748a714c68a2272505e6743bf9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.907599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6afdfc-5a89-4289-8459-0ee9f18560e9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.909216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 592/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/projects/e7dc12748a714c68a2272505e6743bf9 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:41.915421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:42.011242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3765-3c26-44ce-8532-f50d06aa5590 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:42.043868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:42.044397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:42.044678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:42.044953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:42.046315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=e7dc12748a714c68a2272505e6743bf9, user.id=13580e40aca346468fb4c5e31039e485) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:42.046629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:42.048063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ce7ce-3443-4449-a43e-3a26f54d70d4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:42.053330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 593/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/role_assignments?user.id=13580e40aca346468fb4c5e31039e485&scope.project.id=e7dc12748a714c68a2272505e6743bf9&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:43.938074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cabc34-8bd8-44f0-aec7-6ab6aa439801 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:43.938594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cabc34-8bd8-44f0-aec7-6ab6aa439801 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:43.938594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cabc34-8bd8-44f0-aec7-6ab6aa439801 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:43.939887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 593/1186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:43.945286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:43.945727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:43.945849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:43.964562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:43.965752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:43.967923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:44.033584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 594/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:44.303655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cabc34-8bd8-44f0-aec7-6ab6aa439801 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.406612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cabc34-8bd8-44f0-aec7-6ab6aa439801 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.440425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.441049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.441331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.441605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.451650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:44.456444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:44.457564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:45:44.458645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 594/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:44.463828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.562001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1c153-adae-4b1a-b9f1-15985dd664af None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.594980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.595586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.595864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.596133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.596790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:44.597091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:44.598465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:44.603573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:44.606886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 595/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:44.613576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.635782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:44.712392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb723f56-64b5-490d-b6e6-edcf1f609def None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.746545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.747165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.747448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.747722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] PATH_INFO: `/v3/users/13580e40aca346468fb4c5e31039e485` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.748885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] RBAC: Authorizing `identity:get_user(user_id=13580e40aca346468fb4c5e31039e485)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:44.750356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:44.751934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 595/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:44 2022] GET /identity/v3/users/13580e40aca346468fb4c5e31039e485 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:44.757820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.865547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5eec-b6bc-4cec-8c13-364662027aca None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.898359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955beea7-be69-44ef-be47-66790bc27e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.898987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955beea7-be69-44ef-be47-66790bc27e82 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.899268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955beea7-be69-44ef-be47-66790bc27e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.899541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955beea7-be69-44ef-be47-66790bc27e82 None admin] PATH_INFO: `/v3/projects/e7dc12748a714c68a2272505e6743bf9` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.900646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955beea7-be69-44ef-be47-66790bc27e82 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc12748a714c68a2272505e6743bf9)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:44.902093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955beea7-be69-44ef-be47-66790bc27e82 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:44.903711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 596/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:44 2022] GET /identity/v3/projects/e7dc12748a714c68a2272505e6743bf9 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:44.910303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.997246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:45.009295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960c1d3-57ea-4a98-aa85-c0a6c980245d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:45.044148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89c5e-3732-4522-98b3-b46c7af6b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:45.044858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89c5e-3732-4522-98b3-b46c7af6b162 None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:45.045363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89c5e-3732-4522-98b3-b46c7af6b162 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:45.045719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89c5e-3732-4522-98b3-b46c7af6b162 None admin] PATH_INFO: `/v3/projects/e7dc12748a714c68a2272505e6743bf9/users/13580e40aca346468fb4c5e31039e485/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:45.047644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89c5e-3732-4522-98b3-b46c7af6b162 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc12748a714c68a2272505e6743bf9, user_id=13580e40aca346468fb4c5e31039e485, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:45.049100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89c5e-3732-4522-98b3-b46c7af6b162 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:45.119151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 596/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:45:44 2022] PUT /identity/v3/projects/e7dc12748a714c68a2272505e6743bf9/users/13580e40aca346468fb4c5e31039e485/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:45:47.018863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d592-8bce-4fff-a67d-79f9fe01db46 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:47.019410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d592-8bce-4fff-a67d-79f9fe01db46 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:47.019410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d592-8bce-4fff-a67d-79f9fe01db46 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:47.020720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 597/1193] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:47.025802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:47.026220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:47.026384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:47.045272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:47.046505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:47.114603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 597/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:47.395074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d592-8bce-4fff-a67d-79f9fe01db46 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:47.463129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:47.497034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d592-8bce-4fff-a67d-79f9fe01db46 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:47.531687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:47.532312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:47.532593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:47.532868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:47.543421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:47.548344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:47.549518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:45:47.550460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 598/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:47.555601 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:47.637583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:47.659546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49135e-2bd6-4f03-846b-96e412f71d8b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:47.693812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:47.694449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:47.694727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:47.695000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:47.695659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:47.695963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:47.697336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:47.705914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 598/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:47.711298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:47.808607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde0eef-48f2-4dc7-9409-3a72d13c8fac None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:47.833796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:47.841169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:47.842520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:47.842811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:47.843073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] PATH_INFO: `/v3/users/13580e40aca346468fb4c5e31039e485` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:47.844204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] RBAC: Authorizing `identity:get_user(user_id=13580e40aca346468fb4c5e31039e485)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:47.845616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:47.847211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 599/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 13:45:47 2022] GET /identity/v3/users/13580e40aca346468fb4c5e31039e485 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:47.853160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:47.953198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2335c9f-d1e8-4650-8b24-f7a7ed090565 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:47.987901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:47.988754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:47.989035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:47.989309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:48.000122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:48.056421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:48.057882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:48.156333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 599/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:48.162369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:48.262730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217b5b-48f7-422b-8721-1350547675c3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:48.296562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a35000-d7b3-4048-89ce-cce26380cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:48.297187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35000-d7b3-4048-89ce-cce26380cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:48.297475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35000-d7b3-4048-89ce-cce26380cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:48.297751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35000-d7b3-4048-89ce-cce26380cb8e None admin] PATH_INFO: `/v3/projects/e7dc12748a714c68a2272505e6743bf9` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:48.298929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a35000-d7b3-4048-89ce-cce26380cb8e None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc12748a714c68a2272505e6743bf9)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:48.300367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a35000-d7b3-4048-89ce-cce26380cb8e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:48.302029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 600/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 13:45:48 2022] GET /identity/v3/projects/e7dc12748a714c68a2272505e6743bf9 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:48.308214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:48.348997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:48.406591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3223f0-97c1-4c4e-90dd-442a815d993f None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:48.440028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:48.440626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:48.440904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:48.441178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:48.441892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=e7dc12748a714c68a2272505e6743bf9, user.id=13580e40aca346468fb4c5e31039e485) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:48.442220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:48.443639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ac8c-8d12-4141-b221-257ac0f557c8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:48.449280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 600/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:48 2022] GET /identity/v3/role_assignments?user.id=13580e40aca346468fb4c5e31039e485&scope.project.id=e7dc12748a714c68a2272505e6743bf9&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:58.292119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=68 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:58.446738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=48 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:02.627182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7cb02-ca97-4e96-8e63-078cabc2bf17 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:02.627766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7cb02-ca97-4e96-8e63-078cabc2bf17 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:02.627766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7cb02-ca97-4e96-8e63-078cabc2bf17 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:02.629126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 601/1201] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:02.634873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:02.635311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:02.635422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:02.654541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:02.655780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:02.724882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 601/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:03.002026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7cb02-ca97-4e96-8e63-078cabc2bf17 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:03.108976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7cb02-ca97-4e96-8e63-078cabc2bf17 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:03.145500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:03.148083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:03.148083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:03.148083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:03.157330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:03.158875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:03.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:03.174790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:03.175959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:03.175959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:03.175959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:03.175959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 602/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:03.181425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:03.282738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf03f6-365b-4e5b-b6d5-ad8fa5f55852 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:03.311661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:03.317507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:03.318233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:03.318519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:03.318799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:03.319444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:03.319748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:03.321774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51ba5e-4f66-41c0-8db5-96c57239fec7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:03.333294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 602/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:03.339320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:03.394561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:03.441668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7274a-7dd2-45c9-9c40-965a79336d17 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:03.475533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d54f45-3712-46b8-b8e2-4615b40bc23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:03.476178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d54f45-3712-46b8-b8e2-4615b40bc23c None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:03.476455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d54f45-3712-46b8-b8e2-4615b40bc23c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:03.476726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d54f45-3712-46b8-b8e2-4615b40bc23c None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:03.477851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d54f45-3712-46b8-b8e2-4615b40bc23c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:03.479743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d54f45-3712-46b8-b8e2-4615b40bc23c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:03.571557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 603/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 13:47:03 2022] POST /identity/v3/users => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:47:05.557698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f415-2989-4c8e-8a37-464ece9e7b62 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:05.558429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f415-2989-4c8e-8a37-464ece9e7b62 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:05.558429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f415-2989-4c8e-8a37-464ece9e7b62 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:05.559633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 603/1206] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:05.564926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:05.565979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:05.565979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:05.584430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:05.585637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:05.653336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 604/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:05.924526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01f415-2989-4c8e-8a37-464ece9e7b62 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:06.026356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01f415-2989-4c8e-8a37-464ece9e7b62 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:06.061075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:06.061703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:06.062017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:06.062296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:06.073147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:47:06.078048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:06.079214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:06.080332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 604/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:06.085429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:06.125500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:06.186649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8a923-02e4-475a-8b2a-f83d77599685 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:06.223027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:06.223632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:06.223916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:06.224192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:06.224846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:06.225150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:06.226533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:06.235332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 605/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:47:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:06.241131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:06.292529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:06.340624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde188d6-f24a-4681-a32d-fefb854b327d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:06.375124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:06.375766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:06.376070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:06.376345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:06.389095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:06.390581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:06.405859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:06.407063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:06.408180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:06.408180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:06.408180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:06.408180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 605/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:06.413159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:06.513165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:06.516239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dd9ce-d5de-4c2c-82e9-6af716e1b09c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:06.550360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:06.551021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:06.551311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:06.551587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:06.552228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:06.552532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:06.553923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:06.566387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 606/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:06.572265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:06.674982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66e6d-2777-41e4-a935-3ccbb9e7217a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:06.676526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:06.708879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:06.709506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:06.709788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:06.710098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:06.717719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:06.719209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:47:06.731041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:06.732144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:06.733505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 606/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:06.738403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:06.839245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc221-7fe1-4390-8e2d-fa0999776f14 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:06.874102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:06.874749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:06.875027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:06.875297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:06.876012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:06.876338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:06.877788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:06.894857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 607/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 13:47:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:47:06.901250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:07.007386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c9a6f-49f3-4cda-9108-b4550720ef5b None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:07.043816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:07.044423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:07.044703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:07.044975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:07.074770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:07.117344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:07.118857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:07.221353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 607/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:47:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:47:07.228074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:07.247657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:07.325906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fef91-c500-454e-95a1-3883e714bcca None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:07.360393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:07.361001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:07.361282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:07.361558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:07.371485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:07.372849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:07.387785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:07.388946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:07.390106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:07.390106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:07.390106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:07.390106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 608/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:07.394647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:07.455653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:07.495302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589924df-12aa-4f79-8474-446031b7ed31 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:07.532355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:07.533172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:07.533532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:07.533855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:07.534616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:07.534984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:07.536441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:07.548553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 608/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:07.555721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:07.636237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:07.667611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39639ec-39d7-4739-ac36-0d4dc45a7d59 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:07.702534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:07.703182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:07.703469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:07.703745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:07.711198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:07.712658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:47:07.724922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:07.728462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:07.729612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 609/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:07.732465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:07.834462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35500e-e877-414e-a1fd-bba0c24f3530 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:07.856612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:07.868816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:07.869420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:07.869704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:07.870014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:07.870706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:07.871033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:07.872446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961803d-9ee9-49cf-ac2a-1702deec10eb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:07.884613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 609/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:07.890886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:07.991602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadc55e-2fa3-4b5a-910e-0114ad1d9350 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:08.019647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:08.026165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:08.026768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:08.027051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:08.027327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:08.028050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=51e7d81890cc4ee5a357765f2a2bbbe2) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:08.028364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:08.029769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6af2-5575-42a6-89c6-f97b41832b20 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:08.035040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 610/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:47:07 2022] GET /identity/v3/role_assignments?user.id=51e7d81890cc4ee5a357765f2a2bbbe2&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:09.962714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04f77a-62d6-41d8-a817-cb6574f5756e None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:09.963258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04f77a-62d6-41d8-a817-cb6574f5756e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:09.963258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04f77a-62d6-41d8-a817-cb6574f5756e None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:09.964540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 610/1220] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:09.970016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:09.970436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:09.970589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:09.990239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:09.991512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:10.058690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 611/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:47:10.342075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04f77a-62d6-41d8-a817-cb6574f5756e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:10.447646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04f77a-62d6-41d8-a817-cb6574f5756e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:10.481979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:10.482609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:10.483149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:10.483423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:10.493339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:47:10.498122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:10.500775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:10.501812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 611/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:47:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:10.505384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:10.604426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915666e-97ae-4f4d-9375-5ec18b74c424 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:10.629035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:10.640649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:10.641260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:10.641550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:10.641844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:10.642539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:10.642846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:10.644237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:10.652424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 612/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:47:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:10.658262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:10.703544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:10.762435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17552b24-fe6d-4838-988e-fe2a5069b68c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:10.795917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:10.796524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:10.796818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:10.797072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:10.808056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:10.809495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:10.824808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:10.826080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:10.827092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:10.827092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:10.827092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:10.827092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 612/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:10.831986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:10.935828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1eec4-f373-48d2-89a5-1e0349bd9135 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:10.968920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:10.969519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:10.969798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:10.970100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:10.970732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:10.971030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:10.972368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:10.984663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 613/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:10.990513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:11.086940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:11.093632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a737f-3891-4a29-af3b-341ac8e3241c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:11.128329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:11.128958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:11.129237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:11.129513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:11.136982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:11.138479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:47:11.150159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:11.151267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:11.152212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 613/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:11.157231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:11.205446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:11.259068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4de46-de76-401c-99e9-7d3877eb31d5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:11.294167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:11.294799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:11.295066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:11.295347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:11.296074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:11.296381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:11.297791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:11.315720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 614/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 13:47:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:11.321638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:11.424259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41bc6-44fe-4d5c-a9d9-abafc72571ae None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:11.458721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:11.459280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:11.459562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:11.459838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:11.470367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:11.471872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:11.486238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:11.487356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:11.488665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:11.488665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:11.488665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:11.488665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 614/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:11.493837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:11.593035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde3d08-25d6-4f49-9900-9ea92229b6cc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:11.595421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:11.627220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:11.627838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:11.628118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:11.628395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:11.629035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:11.629341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:11.630761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:11.642570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 615/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:11.648955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:11.658279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:11.748820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb461f22-d6e0-4643-96be-29ab7e806bcb None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:11.784371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:11.785000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:11.785279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:11.785551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:11.792395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:11.793829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:47:11.805122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:11.806243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:11.807373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 615/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:11.812190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:11.915564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeba227-eacf-4afc-8310-ef3fb29efc5e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:11.949796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:11.950438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:11.950726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:11.950997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:11.951755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:11.952062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:11.953480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1fc77-94b8-4a9f-bd35-f50502ea0310 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:11.965455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 616/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:11.971652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:12.040326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:12.073825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6911f-94c3-490f-8c2d-77550c621ac3 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:12.106935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27e79f-57e2-4f4e-93ab-7235a6024f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:12.107572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27e79f-57e2-4f4e-93ab-7235a6024f4d None admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:12.108102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27e79f-57e2-4f4e-93ab-7235a6024f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:12.108373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27e79f-57e2-4f4e-93ab-7235a6024f4d None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/51e7d81890cc4ee5a357765f2a2bbbe2/roles/e636c9afecc745f3a419bb17dd3d21b8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:12.123548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27e79f-57e2-4f4e-93ab-7235a6024f4d None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=51e7d81890cc4ee5a357765f2a2bbbe2, role_id=e636c9afecc745f3a419bb17dd3d21b8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:12.124980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27e79f-57e2-4f4e-93ab-7235a6024f4d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:12.202903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 616/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:47:11 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/51e7d81890cc4ee5a357765f2a2bbbe2/roles/e636c9afecc745f3a419bb17dd3d21b8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:47:14.123748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de98b25-f4f6-434f-8576-85fade88e874 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:14.124383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de98b25-f4f6-434f-8576-85fade88e874 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:14.124383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de98b25-f4f6-434f-8576-85fade88e874 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:14.125660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 617/1233] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:14.130864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:14.131337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:14.131447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:14.150558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:14.151863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:14.221452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 617/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:14.497356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de98b25-f4f6-434f-8576-85fade88e874 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:14.522179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:14.598674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de98b25-f4f6-434f-8576-85fade88e874 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:14.633359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:14.634006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:14.634293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:14.634566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:14.644396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:47:14.649878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:14.651156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:14.651988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 618/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 13:47:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:14.657285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:14.703592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:14.758726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca551621-57ad-4d20-9c90-92172c7e0e34 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:14.795256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:14.795865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:14.796147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:14.796421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:14.797190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] RBAC: Adding query filter params (name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:14.797495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:14.798934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:14.807839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 618/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 13:47:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:14.813764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:14.899065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:14.915175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91987644-514f-4826-a7a9-48ba074beb7e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:14.948529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:14.949152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:14.949433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:14.949707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:14.959440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:14.961058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:14.976751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:14.977786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:14.978834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:14.978834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:14.978834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:14.978834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 619/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:14.984184 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:15.085783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755010b-de6f-4395-85d0-a68fdf7ae822 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:15.094908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:15.120363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:15.120957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:15.121233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:15.121502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:15.122159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:15.122459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:15.123851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:15.135457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 619/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:15.141450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:15.243613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466cfe0-6780-4745-8ae7-d312d8936d25 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:15.277263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:15.277887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:15.278241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:15.278480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:15.281219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:15.286293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:15.287745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:47:15.299572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:15.300727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:15.306145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 620/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:15.308266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:15.409524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a19a2-ea2a-4c76-81af-b74c37edddf0 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:15.443946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:15.444549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:15.445727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:15.445727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:15.446062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:15.446366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:15.447785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:15.465371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 620/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 13:47:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:15.471647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:15.574681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a54a6e-1bb0-461f-a97f-6698abd3abac None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:15.609571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:15.610208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:15.610495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:15.610773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:15.673363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:15.680101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:15.681553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:15.779606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 621/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:47:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:47:15.785517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:15.813493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:15.888619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953b3a-8ef6-4943-b18a-15abe2f85500 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:15.924314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:15.924869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:15.925147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:15.925419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:15.936545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:15.938086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:15.953598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:15.954860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:15.955903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:15.955903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:15.955903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:15.955903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 621/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:15.961131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:16.056709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:16.062528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6aaf0b-186f-41ab-bac0-f3fb7b46d104 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:16.097952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:16.098562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:16.098847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:16.099126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:16.099773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:16.100074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:16.101507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:16.112782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 622/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:16.119328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:16.200885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:16.222609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd8667-29f0-4545-9c4b-bc0f97691824 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:16.263743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:16.264391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:16.264674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:16.265010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:16.271918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:16.273748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:47:16.284979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:16.289722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:16.290898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 622/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:16.292927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:16.394046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356705-a2a1-4d51-addf-99b62b1bf0fa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:16.428753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:16.430037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:16.430320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:16.430596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:16.431341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:16.431652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:16.433092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d42d80-3fbc-48c9-8980-a4dd4ff41287 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:16.445600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 623/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:16.451984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:16.554403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca592f4-bd8b-4c3f-af9b-b590a3c190b8 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:16.589315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:16.589943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:16.590226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:16.590501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:16.591224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] RBAC: Adding query filter params (role.id=e636c9afecc745f3a419bb17dd3d21b8, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=51e7d81890cc4ee5a357765f2a2bbbe2) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:16.591533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:16.592977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa80dd-cf9c-4829-8fc2-83f05ccdc9a3 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:16.598896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 623/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:47:16 2022] GET /identity/v3/role_assignments?user.id=51e7d81890cc4ee5a357765f2a2bbbe2&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=e636c9afecc745f3a419bb17dd3d21b8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:18.554141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59c03e-b7ee-4228-9d3b-bb9f601aacef None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:18.554743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59c03e-b7ee-4228-9d3b-bb9f601aacef None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:18.554743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59c03e-b7ee-4228-9d3b-bb9f601aacef None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:18.556055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 624/1247] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:18.561125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:18.561565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:18.561674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:18.568257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:18.581702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:18.583021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:18.655575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 624/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:18.926243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59c03e-b7ee-4228-9d3b-bb9f601aacef None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:18.946051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:19.032325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59c03e-b7ee-4228-9d3b-bb9f601aacef None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:19.065729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:19.066443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:19.066722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:19.067003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:19.079517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:47:19.084363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:19.085810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:19.086995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 625/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:19.091563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:19.192664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed692e-9dc0-471f-adff-895bbdba8b62 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:19.223466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:19.228504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:19.229174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:19.229609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:19.229888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:19.230578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:19.230881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:19.232243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:19.240928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 625/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:19.247265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:19.327569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:19.349955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0ca4b-8d12-4873-b972-5316e193b978 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:19.386395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:19.387039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:19.387322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:19.387594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:19.399128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:19.400581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:19.415791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:19.417013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:19.418132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:19.418132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:19.418132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:19.418132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 626/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:19.423038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:19.525806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90f76e-6d5c-48bc-9703-84dbca85dca4 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:19.560687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:19.561289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:19.561862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:19.561862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:19.562526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:19.562837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:19.564221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:19.576875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 626/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:19.582652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:19.688578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40c64-341a-476d-bb99-f037d8c87b48 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:19.715869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:19.723742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:19.724437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:19.724722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:19.725001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:19.734389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:19.735831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:47:19.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:19.748273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:19.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 627/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:19.754093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:19.810595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:19.855430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e19c13-82cf-44a1-aa11-5c690a55c97c None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:19.890338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:19.890939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:19.891216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:19.891485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:19.892229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:19.892541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:19.893978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:19.911499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 627/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 13:47:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:19.917984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:20.020060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678252cb-edab-4825-ab64-732c4a70280a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:20.053351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:20.054873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:20.055162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:20.055444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:20.114819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:20.128513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:20.129995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:20.233423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 628/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:47:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:47:20.242892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:20.347231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737468-812e-4fb6-929a-2a5de3e7a4e7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:20.361535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:20.383657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:20.384223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:20.384506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:20.384784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:20.395294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:20.396743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:20.412669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:20.418896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:20.420034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:20.420034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:20.420034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:20.420034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 628/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:20.420589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:20.511412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:20.523370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024697-7a3d-427f-b710-92829732bcc3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:20.558165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:20.558827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:20.559328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:20.560029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:20.560694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:20.561020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:20.562417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:20.573880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 629/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:47:20.579674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:20.686587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caeeab-e48e-49a5-af6f-6c64bc4009b5 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:20.720344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:20.720970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:20.721250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:20.721522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:20.728748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:20.730220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:47:20.741964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:20.743173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:20.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 629/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:20.749224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:20.849191 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ca638-59a1-4bc4-8c53-eead6e5e65cc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:20.883011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:20.883606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:20.883890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:20.884162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:20.884856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:20.885145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:20.886576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ec1e-b87c-4371-b73a-5523debbe3dc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:20.898700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 630/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:47:20.904795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:20.930676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:21.007693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822d87f-586e-4621-9b38-0b08c46022cb None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:21.041525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:21.042150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:21.042430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:21.042700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:21.043415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=51e7d81890cc4ee5a357765f2a2bbbe2) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:21.043717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:21.045139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125b53c-0912-4025-aebf-53c6fd57f1a5 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:21.050441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 630/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:47:20 2022] GET /identity/v3/role_assignments?user.id=51e7d81890cc4ee5a357765f2a2bbbe2&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:22.988356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d337c97-fb00-4221-9dda-58007f707b68 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:22.988908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d337c97-fb00-4221-9dda-58007f707b68 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:22.989013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d337c97-fb00-4221-9dda-58007f707b68 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:22.990364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 631/1261] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:22.995712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:22.996146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:22.996275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:23.015226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:23.016423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:23.086925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 631/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:23.364296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d337c97-fb00-4221-9dda-58007f707b68 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:23.375966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:23.466996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d337c97-fb00-4221-9dda-58007f707b68 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:23.502091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:23.502730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:23.503019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:23.503294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:23.514642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:47:23.519544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:23.520688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:23.521758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 632/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:47:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:23.527520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:23.546340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:23.627585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06d38-2759-4f36-af10-cd8c26c8348d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:23.663712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:23.664506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:23.664794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:23.665074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:23.665738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:23.666077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:23.667480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:23.675674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 632/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:47:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:23.681664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:23.748658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:23.781949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adca44-ef6e-4e5f-8818-3aaab3e6b260 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:23.815291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:23.818434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:23.818434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:23.818434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:23.827467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:23.828910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:23.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:23.845189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:23.846296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:23.846296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:23.846296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:23.846296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 633/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:23.851413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:23.935069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:23.956961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9eaef-5dff-4c7f-a105-9261b2e7b997 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:23.992036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:23.992631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:23.992911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:23.993182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:23.993816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:23.994145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:23.995504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:24.008054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 633/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:24.013960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:24.114750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3ed5b-cbbf-471e-bc79-5fe33fc809a2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:24.135636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:24.150407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:24.151037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:24.151325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:24.151602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:24.160246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:24.161749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:47:24.173186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:24.174310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:24.175540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 634/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:47:24.181306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:24.294358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717368a-2bd1-40f0-afad-4c9eff44de0a None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:24.330120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:24.330721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:24.330997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:24.331266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:24.331988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:24.332285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:24.333706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:24.343029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:24.351914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 634/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 13:47:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:24.358768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:24.460489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3fee4-5fb9-4e3c-a2d1-0f5d57cc5fa1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:24.493670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36932913-f356-4374-9d69-87002c31662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:24.494320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36932913-f356-4374-9d69-87002c31662a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:24.494605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36932913-f356-4374-9d69-87002c31662a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:24.494877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36932913-f356-4374-9d69-87002c31662a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:24.505429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36932913-f356-4374-9d69-87002c31662a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:24.508510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36932913-f356-4374-9d69-87002c31662a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-36932913-f356-4374-9d69-87002c31662a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:24.522547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:24.523523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:24.524489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:24.524489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:24.524489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:24.524489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 635/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:24.529505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:24.633238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e7e17-23ad-47d0-8b3e-a1c8cb48e43d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:24.667622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:24.668224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:24.668503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:24.668804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:24.669446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:24.669753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:24.671175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:24.683216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 635/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:24.689053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36932913-f356-4374-9d69-87002c31662a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:24.720375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:24.791864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36932913-f356-4374-9d69-87002c31662a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:24.825702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:24.826356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:24.826640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:24.826916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:24.833531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:24.835149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:47:24.846206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:24.847309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:24.848384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 636/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:24.853945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:24.915838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:24.957087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9216ec-0658-4eca-a707-faddad39d7ec None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:24.993884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:24.994513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:24.994792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:24.995071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:24.995761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:24.996069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:24.997647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2ba10-4a55-4a50-a502-c6bee6aacd8b None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:25.010319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 636/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:25.016470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:25.110750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:25.120455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bf9f3-1f3a-4414-b830-e142a250ab9a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:25.154305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd571-259d-447e-a76d-5c0b1358eaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:25.154975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd571-259d-447e-a76d-5c0b1358eaef None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:25.155258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd571-259d-447e-a76d-5c0b1358eaef None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:25.155534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd571-259d-447e-a76d-5c0b1358eaef None admin] PATH_INFO: `/v3/projects/522bb405d1c44313b55c3c750fa31613/users/51e7d81890cc4ee5a357765f2a2bbbe2/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:25.157541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd571-259d-447e-a76d-5c0b1358eaef None admin] RBAC: Authorizing `identity:create_grant(project_id=522bb405d1c44313b55c3c750fa31613, user_id=51e7d81890cc4ee5a357765f2a2bbbe2, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:25.159014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd571-259d-447e-a76d-5c0b1358eaef None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:25.278819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 637/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 13:47:25 2022] PUT /identity/v3/projects/522bb405d1c44313b55c3c750fa31613/users/51e7d81890cc4ee5a357765f2a2bbbe2/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:47:27.191311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c464d-1a3b-4724-80ea-372af9e75f35 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:27.191857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c464d-1a3b-4724-80ea-372af9e75f35 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:27.191857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c464d-1a3b-4724-80ea-372af9e75f35 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:27.193193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 637/1274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:27.199190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:27.199615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:27.199751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:27.218441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:27.219664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:27.288943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 638/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:27.566370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c464d-1a3b-4724-80ea-372af9e75f35 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:27.670509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c464d-1a3b-4724-80ea-372af9e75f35 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:27.689706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:27.706246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:27.706874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:27.707155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:27.707429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:27.717431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:47:27.722375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:27.723324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:27.726985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 638/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 13:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:27.730153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:27.810512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:27.827830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e8556-1b09-478c-9dba-ec007061a873 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:27.860902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:27.861505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:27.861789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:27.862099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:27.862764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:27.863073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:27.864460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:27.873669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 639/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 13:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:27.879607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:27.981645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f17e0-054a-4130-8e99-4981da6cb2e7 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:28.015659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:28.016310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:28.016590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:28.016865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:28.027985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:28.029424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:28.043996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:28.045233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:28.046394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:28.046394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:28.046394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:28.046394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 639/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:47:28.051186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:28.152604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef5382-e321-4e36-b49f-9877060a54e0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:28.186800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:28.187572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:28.187933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:28.188219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:28.188869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:28.189175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:28.190568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:28.202427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 640/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:28.208209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:28.240602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:28.306903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b3f9-1708-416b-b3af-a2b9cf99d261 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:28.340276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:28.340900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:28.341175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:28.341445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:28.348897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:28.350489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:47:28.361404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:28.374043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:28.375242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 640/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:28.376017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:28.376017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:28.474207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b4319-480c-411c-801f-1bf6c23445bd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:28.508267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:28.508874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:28.509155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:28.509428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:28.510337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:28.510607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:28.522581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:28.529291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 641/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 13:47:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:28.535177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:28.629323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:28.639655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b5ad8-595c-400a-b5cc-e37e457c3773 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:28.673610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:28.674260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:28.674543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:28.674820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:28.743047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:28.744439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:28.831769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:28.843801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 641/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 13:47:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:47:28.849645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:28.914015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:28.953197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08aa38-62ce-4d48-b83f-dcd92b7ab034 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:28.985722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:28.986374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:28.986657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:28.986932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:29.002221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:29.002221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:47:29.016034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:29.017203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:47:29.018404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:47:29.018404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:47:29.018404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:29.018404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 642/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 13:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:29.023114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:29.124720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74444bca-cb42-4f30-95e3-607cf4519fba None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:29.160273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:29.160950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:29.161233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:29.161506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:29.162181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:29.163128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:29.164629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:29.176684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 642/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 13:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:29.183222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:29.285013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc08010-add7-46d2-b9d6-869767cc396d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:29.299354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:29.318961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:29.319668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:29.319945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:29.320220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:29.327590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:29.329030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:47:29.340199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:29.341482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Sep 01 13:47:29.342623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 643/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 13:47:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:29.347233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:29.401164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:29.452910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eac42-d7b8-4152-b836-4d9ee1ec4d59 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:29.488902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:29.489507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:29.489788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:29.490092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:29.490990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:29.491304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:29.492783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1b07f-6f10-4556-9492-c074bb161e10 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:29.505358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 643/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 13:47:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:29.511794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:29.619925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430eb760-2432-4a01-9feb-51ba6cf3eb62 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:29.653356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:29.653984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:29.654271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:29.654550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:29.655253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] RBAC: Adding query filter params (role.id=718611f559c1495c92b36acf3fc8e1fe, scope.project.id=522bb405d1c44313b55c3c750fa31613, user.id=51e7d81890cc4ee5a357765f2a2bbbe2) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:29.655558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:29.656962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a85d0-6b07-44d5-8835-7c01ef9841a1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:29.663215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 644/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 13:47:29 2022] GET /identity/v3/role_assignments?user.id=51e7d81890cc4ee5a357765f2a2bbbe2&scope.project.id=522bb405d1c44313b55c3c750fa31613&role.id=718611f559c1495c92b36acf3fc8e1fe => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:31.570856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16874dab-99e0-4d75-9b2c-ae95fbfad5b1 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:31.571429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16874dab-99e0-4d75-9b2c-ae95fbfad5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:31.571429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16874dab-99e0-4d75-9b2c-ae95fbfad5b1 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:31.572742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 644/1288] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:31.578341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:31.578797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:31.578912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:31.597143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:31.598399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:31.630865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:31.668198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 645/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:47:31.935284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16874dab-99e0-4d75-9b2c-ae95fbfad5b1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:32.039166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16874dab-99e0-4d75-9b2c-ae95fbfad5b1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:32.051264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:32.075319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:32.075980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:32.076263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:32.076537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:32.077117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:32.078532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:32.089750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:32.095056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:32.096079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 645/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:47:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:32.097368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:32.196858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba628ff-09ce-4532-8c56-99df3fa9aa1b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:32.231137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:32.232008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:32.232297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:32.232572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:32.233203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:32.233509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:32.234895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef06b8c-681f-4cce-afa8-ee8ffdfab069 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:32.239968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 646/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 13:47:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:32.246201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:32.347827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc578c0-766b-486f-be5d-9e39b85a505d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:32.382266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:32.382875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:32.383170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:32.383430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] PATH_INFO: `/v3/services` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:32.384055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:32.384357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:32.385718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c28249-613a-41ff-974e-cb9984de337a None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:32.391019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 646/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 13:47:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:34.329402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19165001-50b2-4413-9dc3-3f4fdac6f2c6 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:34.329994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19165001-50b2-4413-9dc3-3f4fdac6f2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:34.329994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19165001-50b2-4413-9dc3-3f4fdac6f2c6 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:34.331311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 647/1293] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:34.336520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd52b7-83ac-48d7-97bd-e838047822d2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:34.336919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd52b7-83ac-48d7-97bd-e838047822d2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:34.337079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd52b7-83ac-48d7-97bd-e838047822d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:34.356008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f2dd52b7-83ac-48d7-97bd-e838047822d2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:34.357468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f2dd52b7-83ac-48d7-97bd-e838047822d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:34.390048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:34.428846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 647/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:34.699845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19165001-50b2-4413-9dc3-3f4fdac6f2c6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:34.731789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:34.801405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19165001-50b2-4413-9dc3-3f4fdac6f2c6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:34.835389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14310282-9424-41e3-a523-ff31f7dabac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:34.835981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14310282-9424-41e3-a523-ff31f7dabac9 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:34.836258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14310282-9424-41e3-a523-ff31f7dabac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:34.836526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14310282-9424-41e3-a523-ff31f7dabac9 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:34.837159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14310282-9424-41e3-a523-ff31f7dabac9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:34.838536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14310282-9424-41e3-a523-ff31f7dabac9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:34.934883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 648/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 13:47:34 2022] POST /identity/v3/services => generated 246 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:47:36.856475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae7866-b9df-401b-94d1-85ddcfdcd8ea None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:36.857039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae7866-b9df-401b-94d1-85ddcfdcd8ea None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:36.857039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae7866-b9df-401b-94d1-85ddcfdcd8ea None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:36.858324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 648/1296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:36.863829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:36.866267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:36.866267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:36.883997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:36.885208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:36.959410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 649/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:37.232954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae7866-b9df-401b-94d1-85ddcfdcd8ea None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:37.344929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae7866-b9df-401b-94d1-85ddcfdcd8ea None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:37.381973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:37.382600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:37.382876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:37.383143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:37.383716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:37.385079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:37.393717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:37.395067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:37.396394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 649/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:47:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:37.401079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:37.419432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:37.501403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f09afd-8051-48bd-bcd4-d321836667c6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:37.535908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:37.536506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:37.537298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:37.537569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:37.538293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:37.538593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:37.539942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:37.544978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 650/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 13:47:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:37.551195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:37.586241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:37.656353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5503d-5e88-4ea6-b401-c365b53ae3d9 None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:37.690891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:37.691491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:37.691772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:37.692049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:37.692788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] RBAC: Adding query filter params (interface=public, service_id=c9e40260ebb84ef886c34f87bde7ac43, region_id=RegionOne) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:37.693125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:37.694532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba42f5-e5e5-42ba-a495-d4d16d6ae514 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:37.699539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 650/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 13:47:37 2022] GET /identity/v3/endpoints?service_id=c9e40260ebb84ef886c34f87bde7ac43&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:37.705282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:37.787768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:37.805606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476926-1e25-4a5b-9c97-6fd9e068827f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:37.840104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b0ced-0c1e-45a0-bc8a-9fdb204d9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:37.840713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0ced-0c1e-45a0-bc8a-9fdb204d9c85 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:37.840994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0ced-0c1e-45a0-bc8a-9fdb204d9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:37.841309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0ced-0c1e-45a0-bc8a-9fdb204d9c85 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:37.841928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b0ced-0c1e-45a0-bc8a-9fdb204d9c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:37.843264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b0ced-0c1e-45a0-bc8a-9fdb204d9c85 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:37.849868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 651/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 13:47:37 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 13:47:39.805786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba643-c535-4c90-83f2-ef426a82cba6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:39.806468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba643-c535-4c90-83f2-ef426a82cba6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:39.806468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba643-c535-4c90-83f2-ef426a82cba6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:39.807743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 651/1302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:47:39.813377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:39.813825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:39.813945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:39.832051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:47:39.833245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:47:39.900741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 652/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:47:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:47:40.173414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba643-c535-4c90-83f2-ef426a82cba6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:40.269889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:40.276854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba643-c535-4c90-83f2-ef426a82cba6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:40.316498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:40.317116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:40.317393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:40.317692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:40.318794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:40.320177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:47:40.329413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:47:40.330817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application with Lock( Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: ERROR keystone.server.flask.application Sep 01 13:47:40.331964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 652/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 13:47:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:47:40.337613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:40.419467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:40.442444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901366bd-3cf5-4c70-bd1b-d47f97bce07a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:40.477864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:40.478503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:40.478788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:40.479063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:40.479745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:47:40.480054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:40.481673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dd57e-1c4a-4070-ae53-a3dffc61c866 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:40.487169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 653/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 13:47:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:47:40.493054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:47:40.598219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75c0d-cc22-4303-9e2e-916a63738f7d None admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:47:40.633654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55166937-c804-40a4-8138-a9e98c836bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:47:40.634302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55166937-c804-40a4-8138-a9e98c836bd8 None admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:47:40.634582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55166937-c804-40a4-8138-a9e98c836bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:47:40.634855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55166937-c804-40a4-8138-a9e98c836bd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:47:40.635519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55166937-c804-40a4-8138-a9e98c836bd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:47:40.636886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55166937-c804-40a4-8138-a9e98c836bd8 None admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:47:40.721948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 653/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 13:47:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:47:50.492505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=30 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:47:50.716861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:27.709989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67644-e015-4d2b-91fa-cf4175b9a774 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:27.710837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67644-e015-4d2b-91fa-cf4175b9a774 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:27.710837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67644-e015-4d2b-91fa-cf4175b9a774 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:27.712268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 654/1307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:56:27.718559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c988-b8b7-4d38-8bdc-5d167d331f80 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:27.720585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c988-b8b7-4d38-8bdc-5d167d331f80 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:27.721121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c988-b8b7-4d38-8bdc-5d167d331f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:27.919038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a9e1c988-b8b7-4d38-8bdc-5d167d331f80 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:27.920616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a9e1c988-b8b7-4d38-8bdc-5d167d331f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:28.056173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 654/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:56:28.388399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dec628-e8fc-4eb3-8480-af9ead98642c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:28.388872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dec628-e8fc-4eb3-8480-af9ead98642c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:28.389211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dec628-e8fc-4eb3-8480-af9ead98642c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:28.391696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 655/1309] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 1 13:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:28.403057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eda62-dca8-401d-ae25-67520bba776b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:28.403862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eda62-dca8-401d-ae25-67520bba776b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:28.404413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eda62-dca8-401d-ae25-67520bba776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:28.579015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d62eda62-dca8-401d-ae25-67520bba776b None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:28.580653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d62eda62-dca8-401d-ae25-67520bba776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:28.609568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:28.734358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 655/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 1 13:56:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:28.745716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dec628-e8fc-4eb3-8480-af9ead98642c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:28.748236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-89dec628-e8fc-4eb3-8480-af9ead98642c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:28.932086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dec628-e8fc-4eb3-8480-af9ead98642c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:29.102320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:29.103690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:29.104341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:29.105232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:29.108929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:29.256145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:29.332831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:29.334768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b25d48-014f-4e3b-b838-bac1e4a3e582 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:29.360671 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 656/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:31.884547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598d09f-2fe3-4972-9085-e060b85a9651 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:31.885360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598d09f-2fe3-4972-9085-e060b85a9651 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:31.885360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598d09f-2fe3-4972-9085-e060b85a9651 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:31.886464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 656/1312] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:31.892611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f87168-f1b7-413d-ac8f-a60353a4bd52 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:31.893508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f87168-f1b7-413d-ac8f-a60353a4bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:31.893508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f87168-f1b7-413d-ac8f-a60353a4bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:31.997241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-97f87168-f1b7-413d-ac8f-a60353a4bd52 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:31.998533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-97f87168-f1b7-413d-ac8f-a60353a4bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:32.147137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 657/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:32.469484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598d09f-2fe3-4972-9085-e060b85a9651 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:32.471276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d598d09f-2fe3-4972-9085-e060b85a9651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:32.655050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598d09f-2fe3-4972-9085-e060b85a9651 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:32.744318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:32.746848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:32.747474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:32.747754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:32.748027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:32.749723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:32.878581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:32.880061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2365f1-8252-4556-b511-138050465a5c service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:32.900575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 657/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:35.548979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8c71-c411-4f7a-9982-b695268d8686 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:35.549951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8c71-c411-4f7a-9982-b695268d8686 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:35.549951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8c71-c411-4f7a-9982-b695268d8686 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:35.550970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 658/1315] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:35.557080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1102be-0a22-4ba5-b32e-82c33cadcfa4 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:35.557770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1102be-0a22-4ba5-b32e-82c33cadcfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:35.557770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1102be-0a22-4ba5-b32e-82c33cadcfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:35.640266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7b1102be-0a22-4ba5-b32e-82c33cadcfa4 None None] MFA Rules not processed for user `c2cca81b77904c7685856e765eb594be`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:35.642349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7b1102be-0a22-4ba5-b32e-82c33cadcfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:35.702308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:35.760363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 658/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:36.048949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8c71-c411-4f7a-9982-b695268d8686 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:36.050672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-992b8c71-c411-4f7a-9982-b695268d8686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:36.060771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:36.178691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8c71-c411-4f7a-9982-b695268d8686 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:36.257970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:36.258592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:36.258871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:36.259145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:36.260742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:36.327008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:36.380821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:36.382354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0df7f-f031-4432-afd0-b986fc0c5b61 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:36.405605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 659/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:39.438628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298d55a-da0b-48c9-b74d-e86dc187a867 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:39.439219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298d55a-da0b-48c9-b74d-e86dc187a867 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:39.439219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298d55a-da0b-48c9-b74d-e86dc187a867 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:39.440549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 659/1318] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:39.446524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83752-3568-4a5a-84d8-1ffad0913645 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:39.447061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83752-3568-4a5a-84d8-1ffad0913645 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:39.447152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83752-3568-4a5a-84d8-1ffad0913645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:39.528907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cbf83752-3568-4a5a-84d8-1ffad0913645 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:39.530293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cbf83752-3568-4a5a-84d8-1ffad0913645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:39.643346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 660/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:39.932813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298d55a-da0b-48c9-b74d-e86dc187a867 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:39.934509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8298d55a-da0b-48c9-b74d-e86dc187a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:40.061193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298d55a-da0b-48c9-b74d-e86dc187a867 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:40.152221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:40.153074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:40.153385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:40.153641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:40.155423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:40.169789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:40.297916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:40.299442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4b58d-5210-4369-8a2c-a99873f1f465 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:40.320134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 660/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:43.966580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec81b-61fb-497f-a91c-8f3e1ba5da0a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:43.967153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec81b-61fb-497f-a91c-8f3e1ba5da0a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:43.967440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec81b-61fb-497f-a91c-8f3e1ba5da0a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:43.970047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 661/1321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:43.979281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c580174-fdb8-4cad-9043-3d936a1dc5a3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:43.980139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c580174-fdb8-4cad-9043-3d936a1dc5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:43.980322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c580174-fdb8-4cad-9043-3d936a1dc5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:44.065997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2c580174-fdb8-4cad-9043-3d936a1dc5a3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:44.067229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2c580174-fdb8-4cad-9043-3d936a1dc5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:44.173083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:44.225786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 661/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:44.532698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ec81b-61fb-497f-a91c-8f3e1ba5da0a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:44.534360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fb5ec81b-61fb-497f-a91c-8f3e1ba5da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:44.565666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:44.741236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ec81b-61fb-497f-a91c-8f3e1ba5da0a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:44.882404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:44.883015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:44.883375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:44.883700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:44.885606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:44.998187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:45.061234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:45.064110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098daa5-0330-4962-b960-d2bc397cf17c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:45.103939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 662/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:47.268272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07231c-67c6-4323-9674-812b8019bfcc None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:47.269158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07231c-67c6-4323-9674-812b8019bfcc None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:47.269317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07231c-67c6-4323-9674-812b8019bfcc None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:47.270981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 662/1324] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:47.277264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6874f5-9454-48fa-8229-b331039291ac None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:47.278059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6874f5-9454-48fa-8229-b331039291ac None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:47.278059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6874f5-9454-48fa-8229-b331039291ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:47.372190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4d6874f5-9454-48fa-8229-b331039291ac None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:47.373422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4d6874f5-9454-48fa-8229-b331039291ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:47.503494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 663/1325] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:47.815498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07231c-67c6-4323-9674-812b8019bfcc None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:47.817090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dd07231c-67c6-4323-9674-812b8019bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:47.949852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07231c-67c6-4323-9674-812b8019bfcc None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:48.054727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:48.055548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:48.055987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:48.056426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:48.058477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:48.081626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:48.235482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:48.238598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05274e99-ede2-404e-bbee-59a53c081cbc service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:48.277166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 663/1326] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:50.361394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864f14-d017-4805-829d-d9d34b181754 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:50.362379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864f14-d017-4805-829d-d9d34b181754 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:50.362379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864f14-d017-4805-829d-d9d34b181754 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:50.364097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 664/1327] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:50.370634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d6f18-dd67-43d9-9730-80c07e5d7825 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:50.371503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d6f18-dd67-43d9-9730-80c07e5d7825 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:50.371693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d6f18-dd67-43d9-9730-80c07e5d7825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:50.468720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1f8d6f18-dd67-43d9-9730-80c07e5d7825 None None] MFA Rules not processed for user `c2cca81b77904c7685856e765eb594be`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:50.470303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1f8d6f18-dd67-43d9-9730-80c07e5d7825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:50.585691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:50.638393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 664/1328] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:50.935914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864f14-d017-4805-829d-d9d34b181754 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:50.938549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-70864f14-d017-4805-829d-d9d34b181754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:50.980274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:51.107070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864f14-d017-4805-829d-d9d34b181754 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:51.216292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:51.216957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:51.217336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:51.218308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:51.220330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:51.366220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:51.420558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:51.423292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9f291-7f37-488b-bfdc-146686765395 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:51.448525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 665/1329] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:54.176111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb599e-2de1-43b8-8842-b8ef2399cca2 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:54.176997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb599e-2de1-43b8-8842-b8ef2399cca2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:54.176997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb599e-2de1-43b8-8842-b8ef2399cca2 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:54.178743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 665/1330] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:56:54.185029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca2ac-8f99-4a0c-8172-2efdcda5a8a1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:54.185625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca2ac-8f99-4a0c-8172-2efdcda5a8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:54.185625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca2ac-8f99-4a0c-8172-2efdcda5a8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:54.271854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-170ca2ac-8f99-4a0c-8172-2efdcda5a8a1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:54.273281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-170ca2ac-8f99-4a0c-8172-2efdcda5a8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:54.396998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 666/1331] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:54.749851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4970d23-7b28-4c96-baf1-63012ee7cced None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:54.750598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4970d23-7b28-4c96-baf1-63012ee7cced None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:54.751015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4970d23-7b28-4c96-baf1-63012ee7cced None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:54.752606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 666/1332] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 1 13:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:54.762022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d441d6-431b-4fee-8bfc-1ec3974f6ccc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:54.762435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d441d6-431b-4fee-8bfc-1ec3974f6ccc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:54.762794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d441d6-431b-4fee-8bfc-1ec3974f6ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:54.820883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:54.863635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-34d441d6-431b-4fee-8bfc-1ec3974f6ccc None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:54.864896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-34d441d6-431b-4fee-8bfc-1ec3974f6ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:54.962459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 667/1333] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 1 13:56:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:56:54.972722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4970d23-7b28-4c96-baf1-63012ee7cced None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:54.974717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e4970d23-7b28-4c96-baf1-63012ee7cced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:55.103929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4970d23-7b28-4c96-baf1-63012ee7cced None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:55.197069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:55.197766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:55.197999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:55.198275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:55.199993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:55.234748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:55.402220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:55.404782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd4ac7-1eac-4562-838b-e36b91f124b3 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:55.442723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 667/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:57.809634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37419da-3414-432a-b2aa-81aaf1bce97b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.810312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37419da-3414-432a-b2aa-81aaf1bce97b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.810312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37419da-3414-432a-b2aa-81aaf1bce97b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.812306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 668/1335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:57.818756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e5f64-3d54-4622-aca5-f1c6513f8612 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.818756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e5f64-3d54-4622-aca5-f1c6513f8612 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.819720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e5f64-3d54-4622-aca5-f1c6513f8612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.899799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-be4e5f64-3d54-4622-aca5-f1c6513f8612 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:57.900946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-be4e5f64-3d54-4622-aca5-f1c6513f8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:57.999140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:58.025567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 668/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:58.322580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37419da-3414-432a-b2aa-81aaf1bce97b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:58.324229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a37419da-3414-432a-b2aa-81aaf1bce97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:58.455818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37419da-3414-432a-b2aa-81aaf1bce97b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:58.492901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:58.554978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:58.555675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:58.555962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:58.556243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:58.558103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:58.726525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:58.729176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf3820-d567-4b9a-b3e7-a5f3af253de7 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:58.769292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 669/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:01.596658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f93bc4-65cc-4019-8508-55fa76eb6953 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:01.597838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f93bc4-65cc-4019-8508-55fa76eb6953 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:01.597838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f93bc4-65cc-4019-8508-55fa76eb6953 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:01.599553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 669/1338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:01.606118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308a80f-8633-478c-ad19-44677f5068f3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:01.606713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308a80f-8633-478c-ad19-44677f5068f3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:01.606713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308a80f-8633-478c-ad19-44677f5068f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:01.687449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2308a80f-8633-478c-ad19-44677f5068f3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:01.688787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2308a80f-8633-478c-ad19-44677f5068f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:01.698907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:01.811709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 670/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:57:02.107038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f93bc4-65cc-4019-8508-55fa76eb6953 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:02.108660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-34f93bc4-65cc-4019-8508-55fa76eb6953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:02.241653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f93bc4-65cc-4019-8508-55fa76eb6953 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:02.363148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:02.363966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:02.364422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:02.364870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:02.368170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:02.422369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:02.513769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:02.515694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6498d-3b4f-4f25-920f-6c39f54e8644 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:02.538016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 670/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:57:07.729505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c18ee-0bce-4db1-a1e9-a63bde8a92ec None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:07.730369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c18ee-0bce-4db1-a1e9-a63bde8a92ec None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:07.730563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c18ee-0bce-4db1-a1e9-a63bde8a92ec None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:07.732133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 671/1341] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:07.738884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca56bb-29f4-428a-9267-efcd65a97bcd None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:07.739471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca56bb-29f4-428a-9267-efcd65a97bcd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:07.739471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca56bb-29f4-428a-9267-efcd65a97bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:07.836174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-90ca56bb-29f4-428a-9267-efcd65a97bcd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:07.837537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-90ca56bb-29f4-428a-9267-efcd65a97bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:07.988199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:08.012254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 671/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:08.316631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c18ee-0bce-4db1-a1e9-a63bde8a92ec None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:08.319576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b44c18ee-0bce-4db1-a1e9-a63bde8a92ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:08.465457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c18ee-0bce-4db1-a1e9-a63bde8a92ec None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:08.537421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:08.601681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:08.602521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:08.602904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:08.603414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:08.605530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:08.760736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:08.762248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731455-1982-4f18-a636-561f184d0896 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:08.784664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 672/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:57:12.464927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173f09d-5b48-4ae7-84aa-56d6e9fdfb9b None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:12.465818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173f09d-5b48-4ae7-84aa-56d6e9fdfb9b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:12.465818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173f09d-5b48-4ae7-84aa-56d6e9fdfb9b None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:12.468030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 672/1344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:57:12.474300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e353899-17bd-41c3-8137-99620fe873ed None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:12.475138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e353899-17bd-41c3-8137-99620fe873ed None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:12.475138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e353899-17bd-41c3-8137-99620fe873ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:12.588079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8e353899-17bd-41c3-8137-99620fe873ed None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:12.590170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8e353899-17bd-41c3-8137-99620fe873ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:12.623423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:12.772871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 673/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:13.124571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173f09d-5b48-4ae7-84aa-56d6e9fdfb9b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:13.127829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e173f09d-5b48-4ae7-84aa-56d6e9fdfb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:13.292275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173f09d-5b48-4ae7-84aa-56d6e9fdfb9b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:13.405269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:13.406273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:13.406787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:13.407319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:13.410483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:13.496150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:13.602303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:13.605351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b00fb-fec8-430e-8205-3623097416e2 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:13.640158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 673/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:19.063013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4d24a-5e23-45fc-9685-43c9dd902428 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:19.063826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4d24a-5e23-45fc-9685-43c9dd902428 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:19.063987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4d24a-5e23-45fc-9685-43c9dd902428 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:19.066393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 674/1347] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:19.072938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7578-923a-4dd7-b313-c862d3cb1e68 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:19.073675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7578-923a-4dd7-b313-c862d3cb1e68 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:19.073675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7578-923a-4dd7-b313-c862d3cb1e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:19.176801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d08b7578-923a-4dd7-b313-c862d3cb1e68 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:19.182522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d08b7578-923a-4dd7-b313-c862d3cb1e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:19.333501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:19.338372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 674/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:57:19.639559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4d24a-5e23-45fc-9685-43c9dd902428 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:19.641241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-48d4d24a-5e23-45fc-9685-43c9dd902428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:19.816611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4d24a-5e23-45fc-9685-43c9dd902428 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:19.889134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:19.932405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:19.932931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:19.933211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:19.933481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:19.935632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:20.114251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:20.115758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dd748-7556-4aa1-ab58-07db74ded324 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:20.146815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 675/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:26.184460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8670f-a757-46dc-8f3e-6349ed53b527 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:26.185573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8670f-a757-46dc-8f3e-6349ed53b527 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:26.185945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8670f-a757-46dc-8f3e-6349ed53b527 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:26.187950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 675/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 13:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:26.199645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23aa50-c3c8-408b-bf01-aede1eb24d93 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:26.200641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23aa50-c3c8-408b-bf01-aede1eb24d93 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:26.200788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23aa50-c3c8-408b-bf01-aede1eb24d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:26.350209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6c23aa50-c3c8-408b-bf01-aede1eb24d93 None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:26.352381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6c23aa50-c3c8-408b-bf01-aede1eb24d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:26.377999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:26.572344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 676/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 13:57:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:31.771763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed18d3-a313-4683-9b6a-ee99b8ed0465 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:31.772318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed18d3-a313-4683-9b6a-ee99b8ed0465 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:31.772704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed18d3-a313-4683-9b6a-ee99b8ed0465 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:31.775464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 676/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 13:57:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:31.782172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6ed7a-3f76-404c-b357-1d88340dee15 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:31.782746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6ed7a-3f76-404c-b357-1d88340dee15 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:31.782998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6ed7a-3f76-404c-b357-1d88340dee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:32.008223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-dbd6ed7a-3f76-404c-b357-1d88340dee15 None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:32.009716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dbd6ed7a-3f76-404c-b357-1d88340dee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:32.105358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:32.161524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 677/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 13:57:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:34.216980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19732fff-0ae7-4f50-80a9-379b3d21ba62 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:34.217658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19732fff-0ae7-4f50-80a9-379b3d21ba62 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:34.218025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19732fff-0ae7-4f50-80a9-379b3d21ba62 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:34.220616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 677/1354] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 13:57:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:34.228907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e26f9b-290f-42f8-a25d-93772434790f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:34.229782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e26f9b-290f-42f8-a25d-93772434790f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:34.230006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e26f9b-290f-42f8-a25d-93772434790f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:34.335998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-26e26f9b-290f-42f8-a25d-93772434790f None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:34.337278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-26e26f9b-290f-42f8-a25d-93772434790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:34.512013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 678/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 13:57:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:57:38.734751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896a4ce-906d-4bbb-ada8-068ac71b3199 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:38.735813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896a4ce-906d-4bbb-ada8-068ac71b3199 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:38.735813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896a4ce-906d-4bbb-ada8-068ac71b3199 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:38.739625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 678/1356] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:38.748190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4560ee-4483-4171-bb76-f12a115c5422 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:38.748949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4560ee-4483-4171-bb76-f12a115c5422 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:38.749478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4560ee-4483-4171-bb76-f12a115c5422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:38.807352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:38.885708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2c4560ee-4483-4171-bb76-f12a115c5422 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:38.889746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2c4560ee-4483-4171-bb76-f12a115c5422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:39.116940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 679/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:39.681511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65e0c-6521-482b-9de7-f8cc0aed90c6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:39.682214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65e0c-6521-482b-9de7-f8cc0aed90c6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:39.682708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65e0c-6521-482b-9de7-f8cc0aed90c6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:39.685096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 679/1358] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 13:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:39.705877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301d369-c503-4e9d-bf3e-212ae3083597 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:39.706609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301d369-c503-4e9d-bf3e-212ae3083597 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:39.707126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301d369-c503-4e9d-bf3e-212ae3083597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:39.811941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2301d369-c503-4e9d-bf3e-212ae3083597 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:39.813453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2301d369-c503-4e9d-bf3e-212ae3083597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:39.889066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:39.985657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 680/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:40.000188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b65e0c-6521-482b-9de7-f8cc0aed90c6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:40.002913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-13b65e0c-6521-482b-9de7-f8cc0aed90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:40.314954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b65e0c-6521-482b-9de7-f8cc0aed90c6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:40.473401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:40.569005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:40.570255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:40.570771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:40.571251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:40.578164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:40.787344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:40.790858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c8fd-2106-4b36-bc6d-7e6b8ccefb5b service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:40.831641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 680/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 832 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:43.653678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560f1b7-091f-496f-99b6-d74fb866b5e7 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:43.654417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560f1b7-091f-496f-99b6-d74fb866b5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:43.654417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560f1b7-091f-496f-99b6-d74fb866b5e7 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:43.655593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 681/1361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:57:43.664657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6125bb7-60d7-4fa9-b2c0-1dd2cb2aa5f9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:43.665378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6125bb7-60d7-4fa9-b2c0-1dd2cb2aa5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:43.665378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6125bb7-60d7-4fa9-b2c0-1dd2cb2aa5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:43.762981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e6125bb7-60d7-4fa9-b2c0-1dd2cb2aa5f9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:43.764449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e6125bb7-60d7-4fa9-b2c0-1dd2cb2aa5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:43.824407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:43.944531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 681/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:44.420835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560f1b7-091f-496f-99b6-d74fb866b5e7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:44.423526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d560f1b7-091f-496f-99b6-d74fb866b5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:44.661985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560f1b7-091f-496f-99b6-d74fb866b5e7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:44.678890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:44.915173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:44.916071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:44.916539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:44.917084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:44.921707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:45.099141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:45.101753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87efd-7a82-448a-b646-3c38a418854e service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:45.131689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 682/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:57:48.231354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbacaf-250c-499c-8c46-2b3e286d0b24 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:48.232010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbacaf-250c-499c-8c46-2b3e286d0b24 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:48.232010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbacaf-250c-499c-8c46-2b3e286d0b24 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:48.233545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 682/1364] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:48.239372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234496-ec9f-4eaf-820e-f8ea9154b08a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:48.240309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234496-ec9f-4eaf-820e-f8ea9154b08a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:48.240590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234496-ec9f-4eaf-820e-f8ea9154b08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:48.262366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:48.292044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14574a23-659f-4189-9176-45267f764433 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:48.292429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14574a23-659f-4189-9176-45267f764433 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:48.293088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14574a23-659f-4189-9176-45267f764433 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:48.295340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 683/1365] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:48.302437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382618-1605-4b82-924a-55081dfcb4f9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:48.303050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382618-1605-4b82-924a-55081dfcb4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:48.303665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382618-1605-4b82-924a-55081dfcb4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:48.334247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3a234496-ec9f-4eaf-820e-f8ea9154b08a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:48.335613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3a234496-ec9f-4eaf-820e-f8ea9154b08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:48.395369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b9382618-1605-4b82-924a-55081dfcb4f9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:48.396744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b9382618-1605-4b82-924a-55081dfcb4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:48.483221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 683/1366] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:48.562032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 684/1367] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:57:48.969184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f197d9-6593-488d-b5a3-a3410cfdff22 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:48.970826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f197d9-6593-488d-b5a3-a3410cfdff22 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:48.972072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f197d9-6593-488d-b5a3-a3410cfdff22 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:48.974923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 684/1368] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 13:57:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:48.985539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77629296-9573-4852-a4b9-bb7b9b580cdb None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:48.986618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77629296-9573-4852-a4b9-bb7b9b580cdb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:48.987177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77629296-9573-4852-a4b9-bb7b9b580cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:49.033299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:49.111946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-77629296-9573-4852-a4b9-bb7b9b580cdb None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:49.113197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-77629296-9573-4852-a4b9-bb7b9b580cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:49.304488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 685/1369] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 13:57:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:49.324238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f197d9-6593-488d-b5a3-a3410cfdff22 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:49.324238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-08f197d9-6593-488d-b5a3-a3410cfdff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:49.455076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:49.538811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f197d9-6593-488d-b5a3-a3410cfdff22 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:49.586578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c65c-162e-4ffb-80dc-9b89e2206343 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:49.586578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c65c-162e-4ffb-80dc-9b89e2206343 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:49.586578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c65c-162e-4ffb-80dc-9b89e2206343 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:49.587867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 686/1370] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 1 13:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:49.593335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:49.593335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:49.594297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:49.707799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] MFA Rules not processed for user `b9f9ea2ca628427bb165e1da9c0d1c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:49.709401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:49.723562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:49.759911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:49.760543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:49.760859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:49.761851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:49.763108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:49.856818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:49.858577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 687/1371] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 1 13:57:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:57:49.858577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:49.870202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ccff-d84b-4642-ba5c-8767d38e0b95 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:49.933334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:49.935670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8df41-5fea-452f-ab9a-aebc27b21fad service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:49.982160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 685/1372] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:49.993081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:49.993713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:49.994022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:49.994297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:49.997091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:50.096576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:50.143029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:50.144700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a355bb-8cd1-4bdb-8770-4ae80a06c2a4 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:50.166840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 688/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:52.365846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cefe13-aca4-4bb7-b6c1-15f5dfb7799d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:52.366670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cefe13-aca4-4bb7-b6c1-15f5dfb7799d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:52.366670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cefe13-aca4-4bb7-b6c1-15f5dfb7799d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:52.367888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 686/1374] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:52.376592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d05cb-7e62-4972-a9d4-260d80ab8f50 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:52.377563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d05cb-7e62-4972-a9d4-260d80ab8f50 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:52.377747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d05cb-7e62-4972-a9d4-260d80ab8f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:52.484730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8e0d05cb-7e62-4972-a9d4-260d80ab8f50 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:52.486702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8e0d05cb-7e62-4972-a9d4-260d80ab8f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:52.635019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 689/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:52.941840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cefe13-aca4-4bb7-b6c1-15f5dfb7799d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:52.943537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-81cefe13-aca4-4bb7-b6c1-15f5dfb7799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:53.186750 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cefe13-aca4-4bb7-b6c1-15f5dfb7799d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:53.246494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:53.368683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:53.369657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:53.369657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:53.369657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:53.371301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:53.509956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:53.512455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61447494-f193-42a0-b23a-f2a3be92c6ef service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:53.541335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 687/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:56.492471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3cf31-db12-4f51-b1d7-3b1df8bd72bd None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:56.493331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3cf31-db12-4f51-b1d7-3b1df8bd72bd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:56.493331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3cf31-db12-4f51-b1d7-3b1df8bd72bd None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:56.494438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 690/1377] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:57:56.503458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618395be-4581-43f6-81c5-7127473eac79 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:56.504306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618395be-4581-43f6-81c5-7127473eac79 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:56.504306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618395be-4581-43f6-81c5-7127473eac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:56.572529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:56.605106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-618395be-4581-43f6-81c5-7127473eac79 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:57:56.610504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-618395be-4581-43f6-81c5-7127473eac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:56.834399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 688/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:57:57.331785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3cf31-db12-4f51-b1d7-3b1df8bd72bd None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:57.333359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-acc3cf31-db12-4f51-b1d7-3b1df8bd72bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:57.393702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:57.578163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3cf31-db12-4f51-b1d7-3b1df8bd72bd None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:57.774447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:57.779672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:57.780308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:57.780602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:57.780853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:57.782685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:57.928379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:57.930072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa2b6e-fae9-499a-b9d1-f711906fc45c service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:57.970984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 691/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:00.764011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5410-1b0a-468d-9acf-6261dfc6724d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:00.765981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5410-1b0a-468d-9acf-6261dfc6724d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:00.765981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5410-1b0a-468d-9acf-6261dfc6724d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:00.767848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 689/1380] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:00.774010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2560f8-556f-4d12-bed7-e640b276bf09 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:00.774672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2560f8-556f-4d12-bed7-e640b276bf09 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:00.774672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2560f8-556f-4d12-bed7-e640b276bf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:00.897272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ae2560f8-556f-4d12-bed7-e640b276bf09 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:00.900346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ae2560f8-556f-4d12-bed7-e640b276bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:01.032554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:01.080087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 692/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:01.460608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5410-1b0a-468d-9acf-6261dfc6724d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:01.463324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-787f5410-1b0a-468d-9acf-6261dfc6724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:01.661454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:01.719204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5410-1b0a-468d-9acf-6261dfc6724d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:01.886345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:01.887990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:01.887990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:01.887990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:01.889653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:02.005030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:02.027403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:02.030285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772798a-2afb-494c-96bd-02bb79be8522 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:02.063507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 690/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:05.133442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee6282-5a59-4567-b31f-f0cb13fd69d0 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:05.134379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee6282-5a59-4567-b31f-f0cb13fd69d0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:05.134379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee6282-5a59-4567-b31f-f0cb13fd69d0 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:05.135653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 693/1383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:05.141975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8897-5e3e-45a2-9c0e-500a1c683bad None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:05.142901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8897-5e3e-45a2-9c0e-500a1c683bad None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:05.142901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8897-5e3e-45a2-9c0e-500a1c683bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:05.228805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-70cd8897-5e3e-45a2-9c0e-500a1c683bad None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:05.231545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-70cd8897-5e3e-45a2-9c0e-500a1c683bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:05.363204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 691/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:05.657042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee6282-5a59-4567-b31f-f0cb13fd69d0 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:05.658647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a3ee6282-5a59-4567-b31f-f0cb13fd69d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:05.898753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee6282-5a59-4567-b31f-f0cb13fd69d0 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:05.991352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:06.080261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:06.080889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:06.081165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:06.081460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:06.083192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:06.202314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:06.203897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d6e39-1c9b-4b74-9a92-da7f390d130c service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:06.227547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 694/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:08.129948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b8b6-c659-48db-815c-68691bafc89b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:08.130783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b8b6-c659-48db-815c-68691bafc89b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:08.130783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b8b6-c659-48db-815c-68691bafc89b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:08.132310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 692/1386] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:08.138807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:08.139764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:08.139764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:08.231144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:08.232628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:08.250954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:08.298612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1fd11-368c-41f3-8227-afffddfe4ba8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:08.299101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1fd11-368c-41f3-8227-afffddfe4ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:08.299487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1fd11-368c-41f3-8227-afffddfe4ba8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:08.301880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 693/1387] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:08.309069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500bbe2-aaf7-448c-bad9-697f1d3b5107 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:08.309417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500bbe2-aaf7-448c-bad9-697f1d3b5107 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:08.309792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500bbe2-aaf7-448c-bad9-697f1d3b5107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:08.388917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 695/1388] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:08.393338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7500bbe2-aaf7-448c-bad9-697f1d3b5107 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:08.394898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7500bbe2-aaf7-448c-bad9-697f1d3b5107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:08.404173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:08.515925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 694/1389] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:08.818552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:08.820121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:08.908725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504f7c6-c7d4-4a11-aaff-a430256aaa14 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:08.909013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504f7c6-c7d4-4a11-aaff-a430256aaa14 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:08.909306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504f7c6-c7d4-4a11-aaff-a430256aaa14 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:08.910741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 695/1390] 10.222.0.25 () {54 vars in 967 bytes} [Thu Sep 1 13:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:08.916254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:08.916534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:08.918152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:08.977508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93f5f8-300a-47a3-80cd-5a117e8b302a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:09.008614 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] MFA Rules not processed for user `4637080dea3a4349b53a3097f4610465`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:09.009878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:09.030094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:09.126827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 696/1391] 10.222.0.25 () {58 vars in 1064 bytes} [Thu Sep 1 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:09.140168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:09.144955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:09.150856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19ec61-c4c8-4618-91b7-db641de9f9e3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:09.157608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:09.157608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:09.157608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:09.157608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:09.158602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:09.265770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:09.266713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:09.267093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:09.267478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:09.269486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:09.332557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:09.335734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574392db-4298-49d1-a8ba-599b6a60df01 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:09.371473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 696/1392] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:09.426275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:09.428066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddad33-ae2f-40f0-b5fb-316017314d9b service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:09.452223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 697/1393] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:10.632423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f18e2-605f-458c-9f2c-f3f0ebe4f5a2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:10.633269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f18e2-605f-458c-9f2c-f3f0ebe4f5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:10.633269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f18e2-605f-458c-9f2c-f3f0ebe4f5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:10.694246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:10.726335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-811f18e2-605f-458c-9f2c-f3f0ebe4f5a2 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:10.727746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-811f18e2-605f-458c-9f2c-f3f0ebe4f5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:10.859658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 697/1394] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 13:58:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:10.924668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df773b5-4faa-4f5e-be76-a5e586ab6163 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:10.925484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df773b5-4faa-4f5e-be76-a5e586ab6163 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:10.925484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df773b5-4faa-4f5e-be76-a5e586ab6163 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:10.926800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 698/1395] 10.222.0.25 () {54 vars in 943 bytes} [Thu Sep 1 13:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:10.937714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ae9db-fda8-4aa3-b7ac-3c603049e400 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:10.938368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ae9db-fda8-4aa3-b7ac-3c603049e400 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:10.938839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ae9db-fda8-4aa3-b7ac-3c603049e400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:11.047099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c23ae9db-fda8-4aa3-b7ac-3c603049e400 None None] MFA Rules not processed for user `f0506db33e0e47aaa47a9a3fd9bc2f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:11.049707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c23ae9db-fda8-4aa3-b7ac-3c603049e400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:11.178549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:11.261795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 698/1396] 10.222.0.25 () {58 vars in 1040 bytes} [Thu Sep 1 13:58:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:11.278018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df773b5-4faa-4f5e-be76-a5e586ab6163 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:11.279828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3df773b5-4faa-4f5e-be76-a5e586ab6163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:11.338163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:11.476988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df773b5-4faa-4f5e-be76-a5e586ab6163 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:11.623598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:11.624237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:11.624519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:11.624789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:11.626653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:11.650428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:11.762577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:11.765872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1d7e1-6564-4b31-8995-f419aada89d1 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:11.782273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 699/1397] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 13:58:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 13:58:12.005338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db8f6-4a38-4e39-abbe-45b5a4954755 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:12.006038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db8f6-4a38-4e39-abbe-45b5a4954755 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:12.006173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db8f6-4a38-4e39-abbe-45b5a4954755 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:12.007726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 699/1398] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:12.015149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484c139-ef33-4a84-b69e-ef9cae465e90 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:12.015829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484c139-ef33-4a84-b69e-ef9cae465e90 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:12.016112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484c139-ef33-4a84-b69e-ef9cae465e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:12.119793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3484c139-ef33-4a84-b69e-ef9cae465e90 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:12.121038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3484c139-ef33-4a84-b69e-ef9cae465e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:12.274075 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 700/1399] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:12.663556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66db8f6-4a38-4e39-abbe-45b5a4954755 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:12.667948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b66db8f6-4a38-4e39-abbe-45b5a4954755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:13.031840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66db8f6-4a38-4e39-abbe-45b5a4954755 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:13.088537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:13.232457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:13.233524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:13.234068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:13.234557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:13.238249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:13.410195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:13.412549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05c5e7-7006-4739-b057-6e148500b537 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:13.446112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 700/1400] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:15.855031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fec8d-f278-4886-acb2-321c63f527c7 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:15.856469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fec8d-f278-4886-acb2-321c63f527c7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:15.857210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fec8d-f278-4886-acb2-321c63f527c7 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:15.859975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 701/1401] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:15.867124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c1bbf-8f47-4162-996f-78e89568e7cb None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:15.868323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c1bbf-8f47-4162-996f-78e89568e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:15.869524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c1bbf-8f47-4162-996f-78e89568e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:15.943959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:15.963516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1d5c1bbf-8f47-4162-996f-78e89568e7cb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:15.964905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1d5c1bbf-8f47-4162-996f-78e89568e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:16.161049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 701/1402] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:16.493023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fec8d-f278-4886-acb2-321c63f527c7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:16.494798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-413fec8d-f278-4886-acb2-321c63f527c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:16.499160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:16.753391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fec8d-f278-4886-acb2-321c63f527c7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:16.875412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:16.924571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:16.925219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:16.925487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:16.925821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:16.927596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:17.062136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:17.063848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f52e2c-1f85-433c-b13f-bea6a51edcbe service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:17.088966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 702/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:19.770399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bdad8-aa08-4d40-9841-24c79fb60016 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:19.771283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bdad8-aa08-4d40-9841-24c79fb60016 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:19.771283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bdad8-aa08-4d40-9841-24c79fb60016 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:19.773356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 702/1404] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:19.779043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35260c94-5de1-4bbd-a841-b4f37c23b8ab None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:19.779906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35260c94-5de1-4bbd-a841-b4f37c23b8ab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:19.779906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35260c94-5de1-4bbd-a841-b4f37c23b8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:19.891028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-35260c94-5de1-4bbd-a841-b4f37c23b8ab None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:19.893527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-35260c94-5de1-4bbd-a841-b4f37c23b8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:19.957554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:20.038802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 703/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:20.588412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bdad8-aa08-4d40-9841-24c79fb60016 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:20.590789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-496bdad8-aa08-4d40-9841-24c79fb60016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:20.824644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:20.880658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bdad8-aa08-4d40-9841-24c79fb60016 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:21.100400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:21.101395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:21.101948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:21.102475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:21.107008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:21.295560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:21.312884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:21.315344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fe98a-7f17-4839-b82e-9a59fcc04a9a service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:21.354298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 703/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:24.186973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a0ce-673f-466f-9ae9-a80fa0a62e62 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:24.189656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a0ce-673f-466f-9ae9-a80fa0a62e62 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:24.190660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a0ce-673f-466f-9ae9-a80fa0a62e62 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:24.193379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 704/1407] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:58:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:24.204054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f25a0-a2e3-4e06-920e-fd62e5d0c70e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:24.204973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f25a0-a2e3-4e06-920e-fd62e5d0c70e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:24.204973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f25a0-a2e3-4e06-920e-fd62e5d0c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:24.347860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4b3f25a0-a2e3-4e06-920e-fd62e5d0c70e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:24.350056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4b3f25a0-a2e3-4e06-920e-fd62e5d0c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:24.567510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 704/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:24.971462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a0ce-673f-466f-9ae9-a80fa0a62e62 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:24.974913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1594a0ce-673f-466f-9ae9-a80fa0a62e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:25.269359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a0ce-673f-466f-9ae9-a80fa0a62e62 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:25.320631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:25.476870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:25.477508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:25.478490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:25.478881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:25.481034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:25.638368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:25.640383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc22d2e-d754-494d-8c76-fcfbcc132f86 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:25.669534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 705/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:34.564695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=73 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:35.665258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=87 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:40.967441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3551c9-c89b-4366-82af-27e684da78a8 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:40.968253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3551c9-c89b-4366-82af-27e684da78a8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:40.968253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3551c9-c89b-4366-82af-27e684da78a8 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:40.969865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 705/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 1 13:58:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:40.976263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36511e4-30bf-4df4-a26e-c5219f0213fd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:40.977341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36511e4-30bf-4df4-a26e-c5219f0213fd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:40.977341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36511e4-30bf-4df4-a26e-c5219f0213fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:41.076118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d36511e4-30bf-4df4-a26e-c5219f0213fd None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:41.077391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d36511e4-30bf-4df4-a26e-c5219f0213fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:41.209838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 706/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 1 13:58:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:41.227181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7322d-c0a0-4851-b427-4792cd68bf33 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:41.227662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7322d-c0a0-4851-b427-4792cd68bf33 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:41.227885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7322d-c0a0-4851-b427-4792cd68bf33 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:41.229358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 706/1412] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 1 13:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:41.235619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e298-0c60-464b-a172-7dd6a0882cb4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:41.236054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e298-0c60-464b-a172-7dd6a0882cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:41.236398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e298-0c60-464b-a172-7dd6a0882cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:41.330613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:41.334321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c4b7e298-0c60-464b-a172-7dd6a0882cb4 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:41.335725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c4b7e298-0c60-464b-a172-7dd6a0882cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:41.482120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 707/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 1 13:58:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:44.443699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4e79-e656-4784-9def-907561a56b89 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:44.444317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4e79-e656-4784-9def-907561a56b89 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:44.444317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4e79-e656-4784-9def-907561a56b89 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:44.446233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:44.452910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0219b-f389-4bb2-a3fb-ed95056254e9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:44.453811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0219b-f389-4bb2-a3fb-ed95056254e9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:44.453811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0219b-f389-4bb2-a3fb-ed95056254e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:44.537184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b1c0219b-f389-4bb2-a3fb-ed95056254e9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:44.538530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b1c0219b-f389-4bb2-a3fb-ed95056254e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:44.615786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:44.687406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:44.983895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb4e79-e656-4784-9def-907561a56b89 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:44.986720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6edb4e79-e656-4784-9def-907561a56b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:45.201761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb4e79-e656-4784-9def-907561a56b89 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:45.331775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:45.411264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:45.412326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:45.412809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:45.413292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:45.417784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:45.560204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:45.562273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbb4c7-c81d-4615-92ac-1712cea1d29c service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:45.584961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 708/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:46.886342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e19f09-ff5c-46b0-a71f-a489fb8753a2 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:46.886942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e19f09-ff5c-46b0-a71f-a489fb8753a2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:46.886942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e19f09-ff5c-46b0-a71f-a489fb8753a2 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:46.888228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:46.894014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529de12-b34b-4c45-a495-9b00259f5eb2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:46.894592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529de12-b34b-4c45-a495-9b00259f5eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:46.894592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529de12-b34b-4c45-a495-9b00259f5eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:46.986403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3529de12-b34b-4c45-a495-9b00259f5eb2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:46.987804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3529de12-b34b-4c45-a495-9b00259f5eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:47.023589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:47.122737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:47.280510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e19f09-ff5c-46b0-a71f-a489fb8753a2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:47.282485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-83e19f09-ff5c-46b0-a71f-a489fb8753a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:47.479505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e19f09-ff5c-46b0-a71f-a489fb8753a2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:47.528961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:47.626810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:47.627666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:47.628048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:47.628446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:47.630298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:47.742933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:47.744822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772b916-5b87-40dc-bc39-a3d5f9ad924f service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:47.767329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 710/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:48.785467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69708c-a04e-4c84-adac-79a26cc819ee None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:48.786096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69708c-a04e-4c84-adac-79a26cc819ee None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:48.786096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69708c-a04e-4c84-adac-79a26cc819ee None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:48.787440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:48.795801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec113-cd47-4b5f-9a7a-baae5a0857bc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:48.796394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec113-cd47-4b5f-9a7a-baae5a0857bc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:48.796520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec113-cd47-4b5f-9a7a-baae5a0857bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:48.862481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:48.889218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ec7ec113-cd47-4b5f-9a7a-baae5a0857bc None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:48.890515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec7ec113-cd47-4b5f-9a7a-baae5a0857bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:49.012964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:49.172347 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69708c-a04e-4c84-adac-79a26cc819ee None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:49.174118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dc69708c-a04e-4c84-adac-79a26cc819ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:49.356926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69708c-a04e-4c84-adac-79a26cc819ee None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:49.372431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:49.493997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:49.494629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:49.494914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:49.495196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:49.496917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:49.650077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:49.653112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225da81-d67e-4e5e-ad06-f38a5cfde5c5 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:49.688850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 711/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:50.694894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db125fe4-9ee3-4873-9eed-4b689274fefd None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:50.695522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db125fe4-9ee3-4873-9eed-4b689274fefd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:50.695522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db125fe4-9ee3-4873-9eed-4b689274fefd None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:50.697103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:50.703268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815ec6e-470c-47e8-a4a8-c6cd73b411f2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:50.703953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815ec6e-470c-47e8-a4a8-c6cd73b411f2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:50.703953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815ec6e-470c-47e8-a4a8-c6cd73b411f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:50.727493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:50.799495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3815ec6e-470c-47e8-a4a8-c6cd73b411f2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:50.800756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3815ec6e-470c-47e8-a4a8-c6cd73b411f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:50.918867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:51.087507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db125fe4-9ee3-4873-9eed-4b689274fefd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:51.089142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-db125fe4-9ee3-4873-9eed-4b689274fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:51.225226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:51.269544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db125fe4-9ee3-4873-9eed-4b689274fefd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:51.426034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:51.426659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:51.426944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:51.427222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:51.428952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:51.522749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:51.548407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:51.551462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace961c-cca1-4943-9e7f-b090aef7728c service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:51.587200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 713/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:52.572026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4c4ce-79ea-46cd-bd56-e96a7393edc3 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:52.572931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4c4ce-79ea-46cd-bd56-e96a7393edc3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:52.572931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4c4ce-79ea-46cd-bd56-e96a7393edc3 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:52.574857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:52.581371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36087579-e778-4412-8b50-c2c6bec123d0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:52.582012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36087579-e778-4412-8b50-c2c6bec123d0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:52.582012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36087579-e778-4412-8b50-c2c6bec123d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:52.684011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-36087579-e778-4412-8b50-c2c6bec123d0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:52.685289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-36087579-e778-4412-8b50-c2c6bec123d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:52.803107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:52.965462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4c4ce-79ea-46cd-bd56-e96a7393edc3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:52.967002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f7e4c4ce-79ea-46cd-bd56-e96a7393edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:53.072788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:53.149763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4c4ce-79ea-46cd-bd56-e96a7393edc3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:53.300292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:53.300836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:53.301123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:53.301405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:53.303300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:53.371743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:53.425263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:53.427345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01059e41-fc2d-444f-bdfe-0a903453544a service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:53.451404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 714/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:54.431686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b1a8f-0f6c-42e5-9cd7-cee6290b9908 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:54.432304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b1a8f-0f6c-42e5-9cd7-cee6290b9908 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:54.432702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b1a8f-0f6c-42e5-9cd7-cee6290b9908 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:54.435403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:54.442143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1241f-0b5e-44f4-aedd-63290ac9269f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:54.442815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1241f-0b5e-44f4-aedd-63290ac9269f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:54.443117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1241f-0b5e-44f4-aedd-63290ac9269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:54.572224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-79c1241f-0b5e-44f4-aedd-63290ac9269f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:54.573520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-79c1241f-0b5e-44f4-aedd-63290ac9269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:54.726708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:54.914509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b1a8f-0f6c-42e5-9cd7-cee6290b9908 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:54.916731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a1b1a8f-0f6c-42e5-9cd7-cee6290b9908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:54.981252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:55.129441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b1a8f-0f6c-42e5-9cd7-cee6290b9908 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:55.270288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:55.270911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:55.271199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:55.271472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:55.274391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:55.282651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:55.419014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:55.422174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82453b0-9448-4dfa-98da-327e62a0475e service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:55.458712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 716/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:56.412491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e67db-1f6d-4d5d-9ddc-645e4b536c65 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:56.413203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e67db-1f6d-4d5d-9ddc-645e4b536c65 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:56.413348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e67db-1f6d-4d5d-9ddc-645e4b536c65 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:56.416252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:56.422510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c75fe3-b70c-4b7b-a1cd-cf6567d6b268 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:56.422849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c75fe3-b70c-4b7b-a1cd-cf6567d6b268 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:56.423306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c75fe3-b70c-4b7b-a1cd-cf6567d6b268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:56.519825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-83c75fe3-b70c-4b7b-a1cd-cf6567d6b268 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:56.521119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-83c75fe3-b70c-4b7b-a1cd-cf6567d6b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:56.617168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:56.659748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:56.823567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e67db-1f6d-4d5d-9ddc-645e4b536c65 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:56.825132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-023e67db-1f6d-4d5d-9ddc-645e4b536c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:56.846397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:57.067357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e67db-1f6d-4d5d-9ddc-645e4b536c65 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:57.200525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:57.235177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:57.236366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:57.236646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:57.236913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:57.238734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:57.385934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:57.385934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5609ef2-6c0f-4b3d-8e4f-540769d79a88 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:57.390986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 717/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:58.417658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344faa-bc40-45a9-8653-e6f2541a4253 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:58.418391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344faa-bc40-45a9-8653-e6f2541a4253 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:58.418391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344faa-bc40-45a9-8653-e6f2541a4253 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:58.420017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:58.426389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a559b8-5bbf-425e-88ce-1f1ae834192b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:58.427246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a559b8-5bbf-425e-88ce-1f1ae834192b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:58.427246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a559b8-5bbf-425e-88ce-1f1ae834192b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:58.527673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-73a559b8-5bbf-425e-88ce-1f1ae834192b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:58.528939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-73a559b8-5bbf-425e-88ce-1f1ae834192b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:58.591697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:58.666859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:58.824530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344faa-bc40-45a9-8653-e6f2541a4253 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:58.826401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6b344faa-bc40-45a9-8653-e6f2541a4253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:59.048963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344faa-bc40-45a9-8653-e6f2541a4253 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:59.131359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:59.223478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:59.224103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:59.224379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:59.224650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:59.227107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:59.335648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:59.337391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f99124-38df-49f5-a429-838349a6d751 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:59.361985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 719/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:59:00.914807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af130e-6d39-4c2f-ad1b-fc3c64bdbd87 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:00.915474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af130e-6d39-4c2f-ad1b-fc3c64bdbd87 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:00.915474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af130e-6d39-4c2f-ad1b-fc3c64bdbd87 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:00.916687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:00.924998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853069a-7fbc-45da-b947-d3e566405211 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:00.925690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853069a-7fbc-45da-b947-d3e566405211 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:00.926486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853069a-7fbc-45da-b947-d3e566405211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:01.065861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b853069a-7fbc-45da-b947-d3e566405211 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:01.067211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b853069a-7fbc-45da-b947-d3e566405211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:01.070852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:01.193995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:59:01.362208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af130e-6d39-4c2f-ad1b-fc3c64bdbd87 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:01.363924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-87af130e-6d39-4c2f-ad1b-fc3c64bdbd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:01.551713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af130e-6d39-4c2f-ad1b-fc3c64bdbd87 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:01.586489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:01.723500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:01.724240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:01.724597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:01.724918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:01.726811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:01.861163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:01.864311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8867e9a-cbb6-43d3-b798-8cc8f9a91085 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:01.902258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 720/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:59:03.973177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87348c2-8804-4469-8292-ece7ce5066fe None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:03.973771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87348c2-8804-4469-8292-ece7ce5066fe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:03.973771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87348c2-8804-4469-8292-ece7ce5066fe None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:03.975982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:03.982391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ce59-24a9-4638-925a-739fc1b5f42b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:03.983071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ce59-24a9-4638-925a-739fc1b5f42b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:03.983071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ce59-24a9-4638-925a-739fc1b5f42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:04.035699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:04.067441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ae66ce59-24a9-4638-925a-739fc1b5f42b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:04.068704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ae66ce59-24a9-4638-925a-739fc1b5f42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:04.205495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:04.504108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87348c2-8804-4469-8292-ece7ce5066fe None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:04.505749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f87348c2-8804-4469-8292-ece7ce5066fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:04.645522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87348c2-8804-4469-8292-ece7ce5066fe None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:04.688127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:04.767741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:04.768658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:04.769148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:04.769625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:04.773316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:04.923646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:04.926318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b0389-7f7b-480d-b665-3c781ad42521 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:04.964462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 722/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:07.972278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a5022-eb03-4074-ac8e-c1d70f5660ac None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:07.973002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a5022-eb03-4074-ac8e-c1d70f5660ac None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:07.973002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a5022-eb03-4074-ac8e-c1d70f5660ac None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:07.974280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:07.983457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9a564-6003-41c4-9d7d-7c3713f3debd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:07.987641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9a564-6003-41c4-9d7d-7c3713f3debd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:07.987641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9a564-6003-41c4-9d7d-7c3713f3debd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:08.071331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-54d9a564-6003-41c4-9d7d-7c3713f3debd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:08.072593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54d9a564-6003-41c4-9d7d-7c3713f3debd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:08.089382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:08.213496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:08.516119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a5022-eb03-4074-ac8e-c1d70f5660ac None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:08.519000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-344a5022-eb03-4074-ac8e-c1d70f5660ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:08.680439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a5022-eb03-4074-ac8e-c1d70f5660ac None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:08.706529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:08.796197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:08.796881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:08.797276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:08.797666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:08.799541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:08.939203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:08.940707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bd27e-2bfe-477a-b77f-17ab25569231 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:08.963004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 723/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:59:11.686323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bf959-dca5-44d4-b518-051db0646781 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:11.687161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bf959-dca5-44d4-b518-051db0646781 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:11.687328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bf959-dca5-44d4-b518-051db0646781 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:11.688836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:59:11.696512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9034-7acf-45a0-975d-1cae8b240577 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:11.697153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9034-7acf-45a0-975d-1cae8b240577 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:11.697153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9034-7acf-45a0-975d-1cae8b240577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:11.763429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:11.789568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-21ba9034-7acf-45a0-975d-1cae8b240577 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:11.790929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-21ba9034-7acf-45a0-975d-1cae8b240577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:11.912857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:59:12.215877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bf959-dca5-44d4-b518-051db0646781 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:12.217689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b69bf959-dca5-44d4-b518-051db0646781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:12.344718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bf959-dca5-44d4-b518-051db0646781 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:12.392676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:12.434875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:12.435566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:12.435922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:12.436250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:12.438169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:12.570831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:12.572388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8696ca-c563-46bd-9d54-bd08afbce53e service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:12.595421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 725/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:14.852109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9359af-2da7-4d69-9d02-9fd569ebc3eb None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:14.852983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9359af-2da7-4d69-9d02-9fd569ebc3eb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:14.852983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9359af-2da7-4d69-9d02-9fd569ebc3eb None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:14.854204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:59:14.860884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee7943-cc6a-4d57-8d14-a36f008b024e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:14.861437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee7943-cc6a-4d57-8d14-a36f008b024e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:14.861437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee7943-cc6a-4d57-8d14-a36f008b024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:14.952395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0eee7943-cc6a-4d57-8d14-a36f008b024e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:14.953937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0eee7943-cc6a-4d57-8d14-a36f008b024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:14.982690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:15.098878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:15.404018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9359af-2da7-4d69-9d02-9fd569ebc3eb None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:15.405621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1e9359af-2da7-4d69-9d02-9fd569ebc3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:15.538208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9359af-2da7-4d69-9d02-9fd569ebc3eb None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:15.564898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:15.635920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:15.636633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:15.637033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:15.637416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:15.639638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:15.780552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:15.782235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d8dd8-c3f6-48db-a0c7-08c26aadd303 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:15.805517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 726/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:59:18.066602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a510-d0f6-4269-b811-ae4e3b023131 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:18.067502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a510-d0f6-4269-b811-ae4e3b023131 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:18.067502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a510-d0f6-4269-b811-ae4e3b023131 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:18.068752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:59:18.075357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5dfea-79de-46d1-ad46-d602c39762b6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:18.076007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5dfea-79de-46d1-ad46-d602c39762b6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:18.076007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5dfea-79de-46d1-ad46-d602c39762b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:18.159309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:18.164803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-87a5dfea-79de-46d1-ad46-d602c39762b6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:18.166206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-87a5dfea-79de-46d1-ad46-d602c39762b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:18.290896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:18.595765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a510-d0f6-4269-b811-ae4e3b023131 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:18.597474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9f22a510-d0f6-4269-b811-ae4e3b023131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:18.733090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a510-d0f6-4269-b811-ae4e3b023131 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:18.793305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:18.823697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:18.824326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:18.824611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:18.824884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:18.827370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:18.994655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:18.996692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1b59e-5ee9-4db7-a578-6b1141339af6 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:19.030604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 728/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:21.300040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d89f4-f84c-4f96-80f8-06f8ee6b5e76 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:21.300692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d89f4-f84c-4f96-80f8-06f8ee6b5e76 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:21.300692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d89f4-f84c-4f96-80f8-06f8ee6b5e76 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:21.302165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:59:21.308923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf64413-cb5e-4207-b6ea-89be99e23eb2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:21.309768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf64413-cb5e-4207-b6ea-89be99e23eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:21.309768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf64413-cb5e-4207-b6ea-89be99e23eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:21.392208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fdf64413-cb5e-4207-b6ea-89be99e23eb2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:21.393460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fdf64413-cb5e-4207-b6ea-89be99e23eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:21.452276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:21.569122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:21.864046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d89f4-f84c-4f96-80f8-06f8ee6b5e76 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:21.866237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-797d89f4-f84c-4f96-80f8-06f8ee6b5e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:21.997572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d89f4-f84c-4f96-80f8-06f8ee6b5e76 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:22.033082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:22.097550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:22.098204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:22.098482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:22.098758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:22.101207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:22.236516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:22.238051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a123f1-0775-439b-a951-6bd71358033c service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:22.260937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 729/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:24.431447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43636e-d845-43d7-ba93-9a5faa00f48a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:24.432347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43636e-d845-43d7-ba93-9a5faa00f48a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:24.432347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43636e-d845-43d7-ba93-9a5faa00f48a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:24.433374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:59:24.439464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a5360-d9f0-4b72-bc3f-3718da8d0628 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:24.440183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a5360-d9f0-4b72-bc3f-3718da8d0628 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:24.440183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a5360-d9f0-4b72-bc3f-3718da8d0628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:24.526061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ce2a5360-d9f0-4b72-bc3f-3718da8d0628 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:24.527296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ce2a5360-d9f0-4b72-bc3f-3718da8d0628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:24.539381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:24.660138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:25.143786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43636e-d845-43d7-ba93-9a5faa00f48a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:25.145465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bd43636e-d845-43d7-ba93-9a5faa00f48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:25.275410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43636e-d845-43d7-ba93-9a5faa00f48a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:25.347689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:25.371276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:25.371990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:25.372317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:25.372635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:25.374566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:25.544725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:25.547473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90150052-c621-47b1-9938-ea26a7028dd8 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:25.587040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 731/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:59:27.749056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f58f-55dd-4352-a2a3-75e370fc0563 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:27.749741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f58f-55dd-4352-a2a3-75e370fc0563 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:27.749741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f58f-55dd-4352-a2a3-75e370fc0563 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:27.751145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:27.758120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3a8c8-6eec-41b6-a4c4-81676aed4544 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:27.758820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3a8c8-6eec-41b6-a4c4-81676aed4544 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:27.758820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3a8c8-6eec-41b6-a4c4-81676aed4544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:27.855265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a5c3a8c8-6eec-41b6-a4c4-81676aed4544 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:27.856756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a5c3a8c8-6eec-41b6-a4c4-81676aed4544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:27.908008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:27.998659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:59:28.305304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f58f-55dd-4352-a2a3-75e370fc0563 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:28.307487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9d02f58f-55dd-4352-a2a3-75e370fc0563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:28.449424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f58f-55dd-4352-a2a3-75e370fc0563 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:28.498370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:28.569287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:28.570313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:28.570825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:28.571324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:28.574712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:28.728504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:28.730130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15109094-9c30-4681-a72b-8927590644f6 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:28.752053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 732/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:59:31.084339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf97e-c980-4fee-8524-faa3a08f0a0c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:31.084980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf97e-c980-4fee-8524-faa3a08f0a0c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:31.085126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf97e-c980-4fee-8524-faa3a08f0a0c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:31.086545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:31.094310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba2b78-33df-4ca8-b621-f0b15f11b615 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:31.094901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba2b78-33df-4ca8-b621-f0b15f11b615 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:31.095336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba2b78-33df-4ca8-b621-f0b15f11b615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:31.229611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-daba2b78-33df-4ca8-b621-f0b15f11b615 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:31.231703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-daba2b78-33df-4ca8-b621-f0b15f11b615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:31.253753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:31.353491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:59:31.649131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bf97e-c980-4fee-8524-faa3a08f0a0c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:31.650946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bd1bf97e-c980-4fee-8524-faa3a08f0a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:31.793198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bf97e-c980-4fee-8524-faa3a08f0a0c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:31.880275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:31.889016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:31.889772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:31.890174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:31.890537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:31.892610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:32.041615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:32.043350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c936-4cf1-4979-a98f-ad7c334297c2 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:32.065765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 734/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:34.263557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67523250-3ce7-4809-87b0-96c03f45ea3f None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:34.264428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67523250-3ce7-4809-87b0-96c03f45ea3f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:34.264428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67523250-3ce7-4809-87b0-96c03f45ea3f None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:34.265522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:34.272520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06ae81-8815-434b-927d-10e66e21ca19 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:34.273205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06ae81-8815-434b-927d-10e66e21ca19 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:34.273205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06ae81-8815-434b-927d-10e66e21ca19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:34.357868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4a06ae81-8815-434b-927d-10e66e21ca19 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:34.359206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4a06ae81-8815-434b-927d-10e66e21ca19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:34.420663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:34.483132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:59:34.798178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67523250-3ce7-4809-87b0-96c03f45ea3f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:34.803333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-67523250-3ce7-4809-87b0-96c03f45ea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:34.937146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67523250-3ce7-4809-87b0-96c03f45ea3f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:35.007663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:35.040334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:35.040962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:35.041301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:35.041569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:35.043363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:35.186513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:35.188013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431f21a-59cb-446e-8876-f11457322340 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:35.210955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 735/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:44.481249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=22 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:45.207433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=66 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:56.692737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef2d8-046d-4bb8-aa24-9f5edc76d7ac None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:56.693534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef2d8-046d-4bb8-aa24-9f5edc76d7ac None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:56.693534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef2d8-046d-4bb8-aa24-9f5edc76d7ac None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:56.695144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 736/1471] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 13:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:56.701686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20e4f0-3eef-4d87-b4ba-f1d66c50f6ba None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:56.702392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20e4f0-3eef-4d87-b4ba-f1d66c50f6ba None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:56.702537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20e4f0-3eef-4d87-b4ba-f1d66c50f6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:56.806221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5a20e4f0-3eef-4d87-b4ba-f1d66c50f6ba None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:56.807567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5a20e4f0-3eef-4d87-b4ba-f1d66c50f6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:56.933938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 736/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 13:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:59:57.256724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d86d2-ec1d-4314-b6a0-316b9a2d2547 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:57.257072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d86d2-ec1d-4314-b6a0-316b9a2d2547 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:57.257480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d86d2-ec1d-4314-b6a0-316b9a2d2547 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:57.259013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 737/1473] 10.222.0.25 () {54 vars in 967 bytes} [Thu Sep 1 13:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:59:57.264744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b62b82-4e6b-4420-8908-42d25cee0673 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:57.265029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b62b82-4e6b-4420-8908-42d25cee0673 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:57.265311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b62b82-4e6b-4420-8908-42d25cee0673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:57.378904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a6b62b82-4e6b-4420-8908-42d25cee0673 None None] MFA Rules not processed for user `4637080dea3a4349b53a3097f4610465`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:59:57.380210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a6b62b82-4e6b-4420-8908-42d25cee0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:57.406146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:57.477319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 737/1474] 10.222.0.25 () {58 vars in 1064 bytes} [Thu Sep 1 13:59:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:59:57.485871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d86d2-ec1d-4314-b6a0-316b9a2d2547 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:59:57.488296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-035d86d2-ec1d-4314-b6a0-316b9a2d2547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:57.631591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d86d2-ec1d-4314-b6a0-316b9a2d2547 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:59:57.726938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:59:57.727866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:59:57.728074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:59:57.728392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:59:57.730328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:59:57.801451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:57.872165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:59:57.875814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b5212-6c2f-4de3-9e3f-f935a1229a90 service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:59:57.898652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 738/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 13:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:00.103763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06d3bd-4190-4c51-bb99-414893ff9ce7 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:00.104366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06d3bd-4190-4c51-bb99-414893ff9ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:00.104482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06d3bd-4190-4c51-bb99-414893ff9ce7 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:00.105839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 738/1476] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 14:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:00.112193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52c4db-5542-42cc-b22b-e5ce8c82a898 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:00.113026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52c4db-5542-42cc-b22b-e5ce8c82a898 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:00.113026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52c4db-5542-42cc-b22b-e5ce8c82a898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:00.189235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fd52c4db-5542-42cc-b22b-e5ce8c82a898 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:00.191160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fd52c4db-5542-42cc-b22b-e5ce8c82a898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:00.307656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 739/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:00.601271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06d3bd-4190-4c51-bb99-414893ff9ce7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:00.604166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8d06d3bd-4190-4c51-bb99-414893ff9ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:00.780547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06d3bd-4190-4c51-bb99-414893ff9ce7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:00.841915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:00.903836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:00.904365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:00.904643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:00.904916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:00.905604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:00.907518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:00.907518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:00:00.907518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:00:00.908700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d01917-6e25-4e8e-b619-6abb2b9d828f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:00.932301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 739/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 14:00:00 2022] GET /identity/v3/projects => generated 2553 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:03.084322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a777d41-6ffb-4d75-9605-341e2604b583 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:03.085245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a777d41-6ffb-4d75-9605-341e2604b583 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:03.085245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a777d41-6ffb-4d75-9605-341e2604b583 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:03.087113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 740/1479] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 14:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:03.093724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7642b8-29c6-45ad-8e26-e588db05f78f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:03.094384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7642b8-29c6-45ad-8e26-e588db05f78f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:03.094384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7642b8-29c6-45ad-8e26-e588db05f78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:03.181398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2e7642b8-29c6-45ad-8e26-e588db05f78f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:03.182708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2e7642b8-29c6-45ad-8e26-e588db05f78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:03.306643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 740/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 14:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:03.825495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a777d41-6ffb-4d75-9605-341e2604b583 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:03.827326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6a777d41-6ffb-4d75-9605-341e2604b583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:03.839997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:04.012214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a777d41-6ffb-4d75-9605-341e2604b583 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:04.139371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:04.197032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:04.197659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:04.197979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:04.198253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:04.200010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:04.329203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:04.331389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ea73f-5917-4f02-a98a-2c23d15133f2 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:04.354655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 741/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:06.594843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338184a-f85f-43b4-9c2e-911c6f3277e6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:06.595735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338184a-f85f-43b4-9c2e-911c6f3277e6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:06.595735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338184a-f85f-43b4-9c2e-911c6f3277e6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:06.598680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 741/1482] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 14:00:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:00:06.605767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a289-9f07-4e57-86cc-5359f1e454d3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:06.606296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a289-9f07-4e57-86cc-5359f1e454d3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:06.606473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a289-9f07-4e57-86cc-5359f1e454d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:06.689216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3f75a289-9f07-4e57-86cc-5359f1e454d3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:06.691175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3f75a289-9f07-4e57-86cc-5359f1e454d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:06.747796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:06.847346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 742/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 14:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:07.144222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338184a-f85f-43b4-9c2e-911c6f3277e6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:07.145978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6338184a-f85f-43b4-9c2e-911c6f3277e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:07.176835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:07.280251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338184a-f85f-43b4-9c2e-911c6f3277e6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:07.371815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:07.372472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:07.372737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:07.373019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:07.374912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:07.600247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:07.669338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:07.670897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b86968-98f1-45a7-8455-397056d8b6c2 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:07.694087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 742/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:10.102492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddb96-e9f6-48a5-a20d-7378265b9e61 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:10.103103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddb96-e9f6-48a5-a20d-7378265b9e61 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:10.103103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddb96-e9f6-48a5-a20d-7378265b9e61 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:10.104457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 743/1485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 14:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:10.110827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffacc91-0f99-440a-9b97-45fca8ea7d15 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:10.111441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffacc91-0f99-440a-9b97-45fca8ea7d15 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:10.111441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffacc91-0f99-440a-9b97-45fca8ea7d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:10.196624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1ffacc91-0f99-440a-9b97-45fca8ea7d15 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:10.198122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1ffacc91-0f99-440a-9b97-45fca8ea7d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:10.319046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 743/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:10.610965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddb96-e9f6-48a5-a20d-7378265b9e61 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:10.613142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a70ddb96-e9f6-48a5-a20d-7378265b9e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:10.745217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddb96-e9f6-48a5-a20d-7378265b9e61 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:10.866865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:10.869259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:10.870176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:10.870572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:10.870944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:10.872777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:11.014690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:11.016181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e1d25-1d91-4832-b71f-7de7c1031368 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:11.041707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 744/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:13.511506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7a1c-17a8-4b9d-a856-27e5260d3d4f None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:13.512167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7a1c-17a8-4b9d-a856-27e5260d3d4f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:13.512167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7a1c-17a8-4b9d-a856-27e5260d3d4f None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:13.513718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 744/1488] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 14:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:13.520108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801e72-b969-4008-8b5e-3ec402b58c0b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:13.520732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801e72-b969-4008-8b5e-3ec402b58c0b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:13.520884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801e72-b969-4008-8b5e-3ec402b58c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:13.603851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a4801e72-b969-4008-8b5e-3ec402b58c0b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:13.605155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a4801e72-b969-4008-8b5e-3ec402b58c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:13.673738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:13.729034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 745/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 14:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:14.030649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf7a1c-17a8-4b9d-a856-27e5260d3d4f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:14.032348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-08bf7a1c-17a8-4b9d-a856-27e5260d3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:14.076296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:14.160374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf7a1c-17a8-4b9d-a856-27e5260d3d4f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:14.250085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:14.250839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:14.251256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:14.251629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:14.254280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:14.352147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:14.393547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:14.395292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859203d-b8de-44b9-b522-38dfe0727bba service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:14.419649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 745/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:17.542079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ab16-c3a1-4c77-8314-fd013008a49e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:17.542735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ab16-c3a1-4c77-8314-fd013008a49e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:17.543066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ab16-c3a1-4c77-8314-fd013008a49e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:17.545747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 746/1491] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 14:00:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:17.552848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe652e-2c50-4d84-8fbe-a4932df73311 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:17.553496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe652e-2c50-4d84-8fbe-a4932df73311 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:17.553496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe652e-2c50-4d84-8fbe-a4932df73311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:17.640283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-67fe652e-2c50-4d84-8fbe-a4932df73311 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:17.641532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-67fe652e-2c50-4d84-8fbe-a4932df73311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:17.770162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 746/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:18.078566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ab16-c3a1-4c77-8314-fd013008a49e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:18.080789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-89b4ab16-c3a1-4c77-8314-fd013008a49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:18.208705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ab16-c3a1-4c77-8314-fd013008a49e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:18.300468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:18.301111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:18.301408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:18.301676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:18.303497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:18.315978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:18.475809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:18.478449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12a5e0-fcc6-42a6-87df-985a80ff7c08 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:18.503985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 747/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:27.766999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=82 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:28.500290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=50 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:56.911444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:56.912359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:56.912359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.013770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:57.015249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:57.147287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 747/1494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:57.156467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.157374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.157374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.180357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:57.181498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:57.190971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 748/1495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:57.196109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:57.207237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbc55b-8489-4efe-9dce-d8de30465a0d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:57.212605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:57.213291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.213620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.214398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.215083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:57.215394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:57.216448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:57.216448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:01:57.216448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:01:57.217361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:57.232111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 748/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:01:57.247883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:57.265306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69426ba5-dc59-47b5-aab0-0e6aa3087be9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:57.273141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:57.274278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.274784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.275271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.277133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:57.280739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:57.280739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:01:57.280739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:01:57.283095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:57.440024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 749/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:57 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:57.444838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:57.456898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b187dc-40e0-4402-b402-433e09e8db28 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:57.520662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:57.528747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:57.529445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.529730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.530037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.530711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:57.531962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:57.531962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:01:57.531962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:01:57.533093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:57.694769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 749/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:57 2022] POST /identity/v3/users => generated 290 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:57.699827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:57.711526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784e720-655e-423a-881c-3510cf808b83 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:57.716886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:57.717577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.717840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.718153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.718818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:57.719851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:57.719851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:01:57.719851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:01:57.720769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:57.736555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 750/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:57.741800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:57.761691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10aef7-ddf1-49b8-ae4c-391a2ae885a2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:57.770195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:57.771282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.771788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.772282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.773444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:57.775517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:57.775517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:01:57.775517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:01:57.777271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ba8b7-7b20-498b-aa7c-c2b521efbcc0 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:57.806122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 750/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:57.811908 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:57.822965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b81724-862d-4674-8494-9e10775d0900 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:57.829349 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:57.833758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.833758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.833758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] PATH_INFO: `/v3/projects/6697e53363c44827a9390c831341e8b1/users/8fb4482ce8ef446882b541d040a29c21/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:57.849160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6697e53363c44827a9390c831341e8b1, user_id=8fb4482ce8ef446882b541d040a29c21, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:57.850334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:57.850334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:01:57.850334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:01:57.852353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89102a-5ed5-47ba-ad9d-43262eef9544 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:57.959034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 751/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:57 2022] PUT /identity/v3/projects/6697e53363c44827a9390c831341e8b1/users/8fb4482ce8ef446882b541d040a29c21/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:57.972911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5febfd-5e64-4ddd-b91f-55653494e514 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:57.973549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5febfd-5e64-4ddd-b91f-55653494e514 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:57.973682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5febfd-5e64-4ddd-b91f-55653494e514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:58.008809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fa5febfd-5e64-4ddd-b91f-55653494e514 None None] MFA Rules not processed for user `8fb4482ce8ef446882b541d040a29c21`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:58.010140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fa5febfd-5e64-4ddd-b91f-55653494e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:58.092019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 751/1502] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:58.107000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016796-a3ea-4d7a-a595-defb35e3db2f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:58.107287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016796-a3ea-4d7a-a595-defb35e3db2f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:58.107566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016796-a3ea-4d7a-a595-defb35e3db2f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:58.108971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 752/1503] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 1 14:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:01:58.114855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e75d-223c-4c6a-b2be-90cae2417a10 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:58.115195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e75d-223c-4c6a-b2be-90cae2417a10 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:58.115614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e75d-223c-4c6a-b2be-90cae2417a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:58.189935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5515e75d-223c-4c6a-b2be-90cae2417a10 None None] MFA Rules not processed for user `b9f9ea2ca628427bb165e1da9c0d1c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:58.191296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5515e75d-223c-4c6a-b2be-90cae2417a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:58.285736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 752/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 1 14:01:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:58.296848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016796-a3ea-4d7a-a595-defb35e3db2f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:58.309163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016796-a3ea-4d7a-a595-defb35e3db2f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:58.313145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:58.313780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:58.314104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:58.314382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:58.325609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:58.327116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:58.338229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 753/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:01:58.391293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1c0f-52cf-4ae9-b4e4-756466944afe None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:58.391632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1c0f-52cf-4ae9-b4e4-756466944afe None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:58.391816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1c0f-52cf-4ae9-b4e4-756466944afe None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:58.393207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 753/1506] 10.222.0.25 () {52 vars in 830 bytes} [Thu Sep 1 14:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:01:58.398509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:58.486203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee8003-b3a8-4aa1-a8cc-b0653faed7e9 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:58.491083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:58.491833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:58.492155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:58.492538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] PATH_INFO: `/v3/users/8fb4482ce8ef446882b541d040a29c21` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:58.494147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb4482ce8ef446882b541d040a29c21)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:58.495794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:01:58.495794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:01:58.495794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:01:58.497553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:58.977551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb4482ce8ef446882b541d040a29c21'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:58.978642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb4482ce8ef446882b541d040a29c21'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:59.008950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb4482ce8ef446882b541d040a29c21'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:59.117610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb4482ce8ef446882b541d040a29c21'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:59.130533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:59.136426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb4482ce8ef446882b541d040a29c21'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:59.143506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 754/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:58 2022] DELETE /identity/v3/users/8fb4482ce8ef446882b541d040a29c21 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:59.148383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:59.148979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:59.149136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:59.171618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:59.172853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:59.183527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:59.244222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 754/1508] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:59.254935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:59.256596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:59.352995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10861cf-2a51-48ec-ac85-f8abe8dbd8d4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:59.358149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e252c-820d-4e48-a667-5d4f33b820d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:59.358796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e252c-820d-4e48-a667-5d4f33b820d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:59.359082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e252c-820d-4e48-a667-5d4f33b820d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:59.359356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e252c-820d-4e48-a667-5d4f33b820d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:59.369763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e252c-820d-4e48-a667-5d4f33b820d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:59.371549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e252c-820d-4e48-a667-5d4f33b820d6 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:59.382897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 755/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:00.256509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:00.258714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:00.274813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2d964-16c3-46bc-bb7a-180ebbb6fbfe None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:00.283848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:00.285096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:00.285586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:00.286104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] PATH_INFO: `/v3/projects/6697e53363c44827a9390c831341e8b1` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:00.288258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6697e53363c44827a9390c831341e8b1)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:00.290400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:02:00.290400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:02:00.290400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:02:00.292219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:00.433379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6697e53363c44827a9390c831341e8b1', 'request_id': 'req-f745159c-1cac-4336-850d-4b35d4bbfb04'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.450327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6697e53363c44827a9390c831341e8b1', 'request_id': 'req-f745159c-1cac-4336-850d-4b35d4bbfb04'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.565490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6697e53363c44827a9390c831341e8b1', 'request_id': 'req-f745159c-1cac-4336-850d-4b35d4bbfb04'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.582149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] The token cache is being invalidate because project 6697e53363c44827a9390c831341e8b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:00.582594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.583739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f745159c-1cac-4336-850d-4b35d4bbfb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.585509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 755/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:00 2022] DELETE /identity/v3/projects/6697e53363c44827a9390c831341e8b1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:09.377598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=38 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:10.577632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:32.469521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ffb48-a9de-46e0-b06c-1250bb5caf5d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:32.470197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ffb48-a9de-46e0-b06c-1250bb5caf5d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:32.470197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ffb48-a9de-46e0-b06c-1250bb5caf5d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:32.471560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:03:32.477975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5dcee-a5f5-4b52-8927-91ca634064a2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:32.478876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5dcee-a5f5-4b52-8927-91ca634064a2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:32.478876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5dcee-a5f5-4b52-8927-91ca634064a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:32.500133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-62f5dcee-a5f5-4b52-8927-91ca634064a2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:32.501395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-62f5dcee-a5f5-4b52-8927-91ca634064a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:32.572375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:32.732422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ffb48-a9de-46e0-b06c-1250bb5caf5d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:32.734213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-758ffb48-a9de-46e0-b06c-1250bb5caf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:32.856105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ffb48-a9de-46e0-b06c-1250bb5caf5d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:32.861851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3070-4f3e-4d35-b5ef-056125582cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:32.862520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3070-4f3e-4d35-b5ef-056125582cb2 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:32.862806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3070-4f3e-4d35-b5ef-056125582cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:32.863081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3070-4f3e-4d35-b5ef-056125582cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:32.872362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b3070-4f3e-4d35-b5ef-056125582cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:32.877437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b3070-4f3e-4d35-b5ef-056125582cb2 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:32.885706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 757/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:33.780598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7aef1-5777-49d8-ace8-16d06114c5c1 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:33.781231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7aef1-5777-49d8-ace8-16d06114c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:33.781404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7aef1-5777-49d8-ace8-16d06114c5c1 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:33.783548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:03:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:03:33.790017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162c24-ec06-4860-9826-c835620f87c9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:33.790680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162c24-ec06-4860-9826-c835620f87c9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:33.790680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162c24-ec06-4860-9826-c835620f87c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:33.812313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0c162c24-ec06-4860-9826-c835620f87c9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:33.813659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0c162c24-ec06-4860-9826-c835620f87c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:33.823335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:33.980443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7aef1-5777-49d8-ace8-16d06114c5c1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:33.996379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7aef1-5777-49d8-ace8-16d06114c5c1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:34.002269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec4230-b2ad-43a1-afa7-2987e75c08bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:34.003035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4230-b2ad-43a1-afa7-2987e75c08bf service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.003263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4230-b2ad-43a1-afa7-2987e75c08bf service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.003537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4230-b2ad-43a1-afa7-2987e75c08bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.013607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec4230-b2ad-43a1-afa7-2987e75c08bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:34.015329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec4230-b2ad-43a1-afa7-2987e75c08bf service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:34.026863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 758/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:34.224365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d5f71-d0e5-4bc6-901a-3a402618b0b7 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.224651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d5f71-d0e5-4bc6-901a-3a402618b0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.224930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d5f71-d0e5-4bc6-901a-3a402618b0b7 None None] PATH_INFO: `/v3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.226345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 759/1517] 10.222.0.72 () {54 vars in 925 bytes} [Thu Sep 1 14:03:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:03:34.233439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862c59f-7480-44e9-b888-ae22d82fbb33 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.233664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862c59f-7480-44e9-b888-ae22d82fbb33 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.233929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862c59f-7480-44e9-b888-ae22d82fbb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.295610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2862c59f-7480-44e9-b888-ae22d82fbb33 None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:34.296914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2862c59f-7480-44e9-b888-ae22d82fbb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:34.378629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 759/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Sep 1 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:38.531739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a0a3-433b-43ae-b01e-a2aeeb8adb4f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:38.532383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a0a3-433b-43ae-b01e-a2aeeb8adb4f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:38.532383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a0a3-433b-43ae-b01e-a2aeeb8adb4f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:38.533730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 760/1519] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 1 14:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:03:38.541009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a29db-5cf2-44ca-9f66-ed692ed48e1b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:38.541582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a29db-5cf2-44ca-9f66-ed692ed48e1b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:38.541827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a29db-5cf2-44ca-9f66-ed692ed48e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:38.574790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-852a29db-5cf2-44ca-9f66-ed692ed48e1b None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:38.576895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-852a29db-5cf2-44ca-9f66-ed692ed48e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:38.592532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 760/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 1 14:03:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:38.605245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a0a3-433b-43ae-b01e-a2aeeb8adb4f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:38.619859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a0a3-433b-43ae-b01e-a2aeeb8adb4f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:38.625535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906e377-8e71-4d5d-bddc-dc6bc1dc7a62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:38.626204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906e377-8e71-4d5d-bddc-dc6bc1dc7a62 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:38.626491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906e377-8e71-4d5d-bddc-dc6bc1dc7a62 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:38.626770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906e377-8e71-4d5d-bddc-dc6bc1dc7a62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:38.636425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906e377-8e71-4d5d-bddc-dc6bc1dc7a62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:38.638104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906e377-8e71-4d5d-bddc-dc6bc1dc7a62 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:38.650489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 761/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:03:45.314282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9add-f97f-4f51-8683-efba815e0bf8 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:45.314912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9add-f97f-4f51-8683-efba815e0bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:45.314912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9add-f97f-4f51-8683-efba815e0bf8 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:45.316919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:03:45.324989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e02522-a2a0-423f-a280-c3aba079e476 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:45.325866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e02522-a2a0-423f-a280-c3aba079e476 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:45.325866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e02522-a2a0-423f-a280-c3aba079e476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:45.352063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-01e02522-a2a0-423f-a280-c3aba079e476 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:45.353443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-01e02522-a2a0-423f-a280-c3aba079e476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:45.362327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:45.527710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b9add-f97f-4f51-8683-efba815e0bf8 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:45.544156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b9add-f97f-4f51-8683-efba815e0bf8 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:45.550346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f80bd-54ab-43c7-99ed-f135961507fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:45.550983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f80bd-54ab-43c7-99ed-f135961507fb service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:45.551457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f80bd-54ab-43c7-99ed-f135961507fb service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:45.551734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f80bd-54ab-43c7-99ed-f135961507fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:45.561456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f80bd-54ab-43c7-99ed-f135961507fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:45.563305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f80bd-54ab-43c7-99ed-f135961507fb service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:45.576287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 762/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:46.464000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafada4a-5c8b-49c0-9bd1-c08f8719ca84 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:46.464707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafada4a-5c8b-49c0-9bd1-c08f8719ca84 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:46.464707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafada4a-5c8b-49c0-9bd1-c08f8719ca84 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:46.465996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:03:46.471779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269bae-690a-4021-8594-4c2bb902d210 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:46.473042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269bae-690a-4021-8594-4c2bb902d210 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:46.473443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269bae-690a-4021-8594-4c2bb902d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:46.496455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c6269bae-690a-4021-8594-4c2bb902d210 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:46.497708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c6269bae-690a-4021-8594-4c2bb902d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:46.507140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:46.673275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafada4a-5c8b-49c0-9bd1-c08f8719ca84 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:46.689974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafada4a-5c8b-49c0-9bd1-c08f8719ca84 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:46.697023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c6f5d-b9cf-4707-abbb-9bfd3eb57fae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:46.697654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c6f5d-b9cf-4707-abbb-9bfd3eb57fae service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:46.697959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c6f5d-b9cf-4707-abbb-9bfd3eb57fae service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:46.698243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c6f5d-b9cf-4707-abbb-9bfd3eb57fae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:46.708493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c6f5d-b9cf-4707-abbb-9bfd3eb57fae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:46.710197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c6f5d-b9cf-4707-abbb-9bfd3eb57fae service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:46.721469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 764/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:51.029801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4d24e-5393-4d84-995b-56b3aeafcb9e None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:51.030753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4d24e-5393-4d84-995b-56b3aeafcb9e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:51.030753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4d24e-5393-4d84-995b-56b3aeafcb9e None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:51.032351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:03:51.037843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec5dad-3f63-4cd4-a174-ab6b3fea5a71 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:51.038778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec5dad-3f63-4cd4-a174-ab6b3fea5a71 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:51.038943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec5dad-3f63-4cd4-a174-ab6b3fea5a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:51.074745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cbec5dad-3f63-4cd4-a174-ab6b3fea5a71 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:51.077367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cbec5dad-3f63-4cd4-a174-ab6b3fea5a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:51.089491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:51.401125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4d24e-5393-4d84-995b-56b3aeafcb9e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:51.404007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-03a4d24e-5393-4d84-995b-56b3aeafcb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:51.446146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:51.481545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4d24e-5393-4d84-995b-56b3aeafcb9e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:51.485982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858448c-df11-455f-b7f4-db00b69f7c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:51.486680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858448c-df11-455f-b7f4-db00b69f7c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:51.487008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858448c-df11-455f-b7f4-db00b69f7c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:51.487328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858448c-df11-455f-b7f4-db00b69f7c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:51.499355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858448c-df11-455f-b7f4-db00b69f7c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:51.501083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858448c-df11-455f-b7f4-db00b69f7c3d service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:51.514367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 765/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:53.593257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638dff-6573-4748-bbb0-9cc8a7cd86bf None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:53.593883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638dff-6573-4748-bbb0-9cc8a7cd86bf None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:53.593883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638dff-6573-4748-bbb0-9cc8a7cd86bf None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:53.595336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:03:53.603699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07053ec4-dd18-4fed-8e9c-4a1d3af7ca40 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:53.604197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07053ec4-dd18-4fed-8e9c-4a1d3af7ca40 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:53.604655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07053ec4-dd18-4fed-8e9c-4a1d3af7ca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:53.634837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-07053ec4-dd18-4fed-8e9c-4a1d3af7ca40 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:53.636945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-07053ec4-dd18-4fed-8e9c-4a1d3af7ca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:53.650062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:53.944239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14638dff-6573-4748-bbb0-9cc8a7cd86bf None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:53.959407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14638dff-6573-4748-bbb0-9cc8a7cd86bf None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:53.964237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff8921-3f2e-456f-b701-5a9c70ffc05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:53.964994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff8921-3f2e-456f-b701-5a9c70ffc05f service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:53.965411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff8921-3f2e-456f-b701-5a9c70ffc05f service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:53.965790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff8921-3f2e-456f-b701-5a9c70ffc05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:53.977943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff8921-3f2e-456f-b701-5a9c70ffc05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:53.979696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff8921-3f2e-456f-b701-5a9c70ffc05f service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:53.992530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 767/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:57.752336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1c92-86e3-4861-9eb7-ed82afa06ab4 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:57.753525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1c92-86e3-4861-9eb7-ed82afa06ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:57.753525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1c92-86e3-4861-9eb7-ed82afa06ab4 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:57.755244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:03:57.761554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdbd2b-52f0-4857-9de3-44cc4e69ebfa None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:57.762214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdbd2b-52f0-4857-9de3-44cc4e69ebfa None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:57.762214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdbd2b-52f0-4857-9de3-44cc4e69ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:57.783768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-31bdbd2b-52f0-4857-9de3-44cc4e69ebfa None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:57.785388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-31bdbd2b-52f0-4857-9de3-44cc4e69ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:57.793710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:58.807079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1c92-86e3-4861-9eb7-ed82afa06ab4 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:58.809800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c63a1c92-86e3-4861-9eb7-ed82afa06ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:58.920372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1c92-86e3-4861-9eb7-ed82afa06ab4 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:58.926604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0020a-ed07-4f7e-a972-ac2da77f080b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:58.927708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0020a-ed07-4f7e-a972-ac2da77f080b service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:58.928200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0020a-ed07-4f7e-a972-ac2da77f080b service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:58.928689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0020a-ed07-4f7e-a972-ac2da77f080b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:58.946396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0020a-ed07-4f7e-a972-ac2da77f080b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:58.949168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0020a-ed07-4f7e-a972-ac2da77f080b service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:58.966492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 768/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:04:01.475368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60927c1f-60c3-45a7-9e6e-07a0dfa36efe None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:01.476006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60927c1f-60c3-45a7-9e6e-07a0dfa36efe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:01.476006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60927c1f-60c3-45a7-9e6e-07a0dfa36efe None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:01.477378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:04:01.486486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0e15b-e226-4e3b-8b99-b206148c9b69 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:01.487672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0e15b-e226-4e3b-8b99-b206148c9b69 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:01.487672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0e15b-e226-4e3b-8b99-b206148c9b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:01.515813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-16a0e15b-e226-4e3b-8b99-b206148c9b69 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:01.517096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-16a0e15b-e226-4e3b-8b99-b206148c9b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:01.527807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:04:02.139607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60927c1f-60c3-45a7-9e6e-07a0dfa36efe None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:02.144534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-60927c1f-60c3-45a7-9e6e-07a0dfa36efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:02.167095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=77 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:02.184241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60927c1f-60c3-45a7-9e6e-07a0dfa36efe None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:02.194531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f0cc5-3a83-428d-8c86-5cf6f9d2ae9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:02.196272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0cc5-3a83-428d-8c86-5cf6f9d2ae9a service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:02.197018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0cc5-3a83-428d-8c86-5cf6f9d2ae9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:02.197814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0cc5-3a83-428d-8c86-5cf6f9d2ae9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:02.215599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f0cc5-3a83-428d-8c86-5cf6f9d2ae9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:02.217131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f0cc5-3a83-428d-8c86-5cf6f9d2ae9a service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:02.229285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 770/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:04:11.509007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=47 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:12.395678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:32.346858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:32.396262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:32.396262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:32.396262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:32.396262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:32.401125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 770/1540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:32.411790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:32.413435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:32.414310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:32.498653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:32.502507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:32.527919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 771/1541] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:32.539789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:32.569874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e62c0-7143-4c09-bdde-d5be9fbf2226 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:32.599940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:32.601166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:32.601660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:32.602183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:32.604133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:32.604133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:32.612439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:32.647891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 771/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:32.665423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:32.692784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacfe79-6b97-4e90-8c2c-d6f6e36d2c10 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:32.709289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:32.710905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:32.711741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:32.712695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:32.714193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:32.718250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:32.964712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 772/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:32 2022] POST /identity/v3/projects => generated 383 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:32.971163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:32.996575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74786457-5ce8-42ec-9969-33dbe6f32d8c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:33.152498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:33.153731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:33.155043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:33.155941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:33.158145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:33.163095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:33.289005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 772/1544] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:32 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:33.295859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:33.327078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0310ce3-26b2-44b4-a1ef-2dbb9c717d08 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:33.338982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:33.340192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:33.340678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:33.341138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:33.342301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:33.346350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:33.369238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 773/1545] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:33.375417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:33.401878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c0bc1-a43b-4637-a695-3ed52171eae8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:33.416687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b077-f2d6-44bc-bd3a-b550f08c29b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:33.418278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b077-f2d6-44bc-bd3a-b550f08c29b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:33.419091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b077-f2d6-44bc-bd3a-b550f08c29b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:33.419905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b077-f2d6-44bc-bd3a-b550f08c29b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:33.421380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b077-f2d6-44bc-bd3a-b550f08c29b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:33.425879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b077-f2d6-44bc-bd3a-b550f08c29b8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:33.457482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 773/1546] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:33.466380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:33.503807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999e83-0b79-4b08-bac5-843d3bd9c338 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:33.517704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71528c7-c9f3-41f9-a870-e36c1a6a37cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:33.523650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71528c7-c9f3-41f9-a870-e36c1a6a37cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:33.524111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71528c7-c9f3-41f9-a870-e36c1a6a37cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:33.524554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71528c7-c9f3-41f9-a870-e36c1a6a37cd admin admin] PATH_INFO: `/v3/projects/238327a3ec7946ada50cc2a71d542415/users/d58fb7cea93a4b46ada7e58657768096/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:33.556506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71528c7-c9f3-41f9-a870-e36c1a6a37cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=238327a3ec7946ada50cc2a71d542415, user_id=d58fb7cea93a4b46ada7e58657768096, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:33.559221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71528c7-c9f3-41f9-a870-e36c1a6a37cd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:33.660206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 774/1547] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:33 2022] PUT /identity/v3/projects/238327a3ec7946ada50cc2a71d542415/users/d58fb7cea93a4b46ada7e58657768096/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:33.709211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e49c3-5cfe-4175-b269-199395ff84bd None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:33.710225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e49c3-5cfe-4175-b269-199395ff84bd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:33.711047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e49c3-5cfe-4175-b269-199395ff84bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:33.788859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a50e49c3-5cfe-4175-b269-199395ff84bd None None] MFA Rules not processed for user `d58fb7cea93a4b46ada7e58657768096`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:33.792108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a50e49c3-5cfe-4175-b269-199395ff84bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:33.827878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1121-025c-4eff-b1af-9e134966b910 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:33.828359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1121-025c-4eff-b1af-9e134966b910 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:33.829259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1121-025c-4eff-b1af-9e134966b910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:33.892336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cc8d1121-025c-4eff-b1af-9e134966b910 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:33.893850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cc8d1121-025c-4eff-b1af-9e134966b910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.000973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 774/1548] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.061600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee9d3-6594-4740-831f-fc5a92421732 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.067516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee9d3-6594-4740-831f-fc5a92421732 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.081250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee9d3-6594-4740-831f-fc5a92421732 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.081250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 775/1549] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 14:05:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:05:34.092581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 775/1550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.098579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cf561-1faa-44ec-95fe-f227da0fee4f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.099578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cf561-1faa-44ec-95fe-f227da0fee4f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.100339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cf561-1faa-44ec-95fe-f227da0fee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.105235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.105518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.105867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.145330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.145330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.164571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 776/1551] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:05:34.170931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:34.193519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379958e2-f331-404a-8583-a4cbf5100af0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:34.212321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:34.212321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.212321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.212321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.213094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:34.213255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:34.215169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:05:34.215169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:05:34.215169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:05:34.216359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed98b-3364-4fb7-935a-1e6cc4b37248 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:34.237790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d58cf561-1faa-44ec-95fe-f227da0fee4f None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.241104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d58cf561-1faa-44ec-95fe-f227da0fee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.241854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 777/1552] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:05:34.249936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.249936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.249936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.301741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.303013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.330068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 778/1553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.331602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:34.357451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5338280-21ff-473c-812b-e1d3ed0f1228 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:34.364745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:34.371537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326433-30fa-48bb-8b13-3d2e84454f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:34.373868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326433-30fa-48bb-8b13-3d2e84454f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.374814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326433-30fa-48bb-8b13-3d2e84454f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.376616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326433-30fa-48bb-8b13-3d2e84454f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.378443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6326433-30fa-48bb-8b13-3d2e84454f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:34.387717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6326433-30fa-48bb-8b13-3d2e84454f58 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:34.506696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 776/1554] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.510326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.510326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.510326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.541420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 779/1555] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/projects => generated 389 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:34.553160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.557639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.560094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.585014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.590238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.636217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.638281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.762042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:34.772695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 777/1556] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.782555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:34.803017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 780/1557] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.805891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:34.820057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347c5f3-489c-4820-b658-b2a87c9db6ac None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:34.825076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:34.825722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.826043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.826313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.827247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:34.829031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:05:34.829031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:05:34.829031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:05:34.830350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef861e1-69d4-4120-86d4-52545c8df576 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:34.882022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe994d-3f6b-495a-877b-4f682edf1ff0 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:34.888460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d63fe-4e5f-413b-8895-40f92da1d856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:34.889101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d63fe-4e5f-413b-8895-40f92da1d856 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.889419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d63fe-4e5f-413b-8895-40f92da1d856 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.889741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d63fe-4e5f-413b-8895-40f92da1d856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.929469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 781/1558] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:34.930925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41182869-591e-4223-9a6e-96d15cb52298 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.931211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41182869-591e-4223-9a6e-96d15cb52298 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.931498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41182869-591e-4223-9a6e-96d15cb52298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.953305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-41182869-591e-4223-9a6e-96d15cb52298 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.958541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41182869-591e-4223-9a6e-96d15cb52298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:34.962359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d63fe-4e5f-413b-8895-40f92da1d856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:34.963874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 782/1559] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:34.964693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d63fe-4e5f-413b-8895-40f92da1d856 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:34.965323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd24017-05aa-49d4-93b9-ffa4dc2b9124 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.965792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd24017-05aa-49d4-93b9-ffa4dc2b9124 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.966060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd24017-05aa-49d4-93b9-ffa4dc2b9124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.980149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 778/1560] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:34.981632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:34.981850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:34.982169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:34.991478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7dd24017-05aa-49d4-93b9-ffa4dc2b9124 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:34.993089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:34.993996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7dd24017-05aa-49d4-93b9-ffa4dc2b9124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.005130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 783/1561] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.006403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5560e-5c49-49e3-8f97-d3e1d7c8dfa3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.006715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5560e-5c49-49e3-8f97-d3e1d7c8dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.007023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5560e-5c49-49e3-8f97-d3e1d7c8dfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.007701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.009207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.022560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 779/1562] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.024866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.037827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c292e-42fa-49f6-900c-62dd1879cf2f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.043326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a5e5560e-5c49-49e3-8f97-d3e1d7c8dfa3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.043326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a5e5560e-5c49-49e3-8f97-d3e1d7c8dfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.043770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.043957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.044316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.044657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.045453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:35.045839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.047728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.059298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 784/1563] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.061571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6bb13-9c59-4860-bfec-3121539be67c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.062116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6bb13-9c59-4860-bfec-3121539be67c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.062598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6bb13-9c59-4860-bfec-3121539be67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.071316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 780/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:35.072382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.089735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e8189-60d2-4712-aa7a-57dd5000cc09 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.093841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-10a6bb13-9c59-4860-bfec-3121539be67c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.095962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-10a6bb13-9c59-4860-bfec-3121539be67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.100719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dde495-dcb3-46eb-93a9-7dc7638c1aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.101805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dde495-dcb3-46eb-93a9-7dc7638c1aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.102271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dde495-dcb3-46eb-93a9-7dc7638c1aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.102788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dde495-dcb3-46eb-93a9-7dc7638c1aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.104155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dde495-dcb3-46eb-93a9-7dc7638c1aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.107583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 785/1565] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.107880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dde495-dcb3-46eb-93a9-7dc7638c1aaa admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.109250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc642-0581-48a1-9f88-2217757239d4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.109485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc642-0581-48a1-9f88-2217757239d4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.109734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc642-0581-48a1-9f88-2217757239d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.135150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 781/1566] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:35.137599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.138220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7d9bc642-0581-48a1-9f88-2217757239d4 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.138413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.138846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.139957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7d9bc642-0581-48a1-9f88-2217757239d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.158298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 786/1567] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.158298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.158298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.159220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.191414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.194175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.198733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.201195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.216363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 782/1568] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.217828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.218477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 787/1569] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.220446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.239080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f8ee-b8db-4723-8484-d14416d4cabe None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.240054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62295f-23a9-4a60-a9f6-62ef614ed4d2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.247646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.247960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.248831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.249080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.249414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.249744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.250024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.251190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.251938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:35.252310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.252483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.254467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:05:35.254467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:05:35.254467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:05:35.255879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d645d44-f51b-4797-8ed1-d0aae1291301 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.257647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.279875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 788/1570] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:35.282601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.283232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.283788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.316966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:35.319020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:35.333389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 789/1571] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:35.336385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.350258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b908a37-c8b0-4f5c-b855-9e8e723c3139 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.356376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.357234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.357700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.358128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.359101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.360980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.375868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 790/1572] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:35.378029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.400216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218c579-5952-43fa-a36e-9c99284951b2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.402571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 783/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:35.404415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.491118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.491947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.492312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.492688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.493465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:35.493846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.495584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.510267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 791/1574] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:35.511268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.529108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a77711-5c50-437e-8589-caf92beed0e6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.534230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559a968-5c1c-445c-8733-3016b9eb0605 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.542101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.543548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.544201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.545036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.546592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:35.547126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.547712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.548093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.548561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.548911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.549714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:35.550081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.551802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.552647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.566603 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 792/1575] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:35.567538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.581365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 784/1576] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:35.582442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.587235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871cee3-5f1e-4f85-8f50-6562d8286bdf admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.599374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2011c0-4a0e-4a29-b6d0-b51f148501f9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.601945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba984fa-e9c6-4ee2-a49b-5fc8b573aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.602903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba984fa-e9c6-4ee2-a49b-5fc8b573aa87 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.603357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba984fa-e9c6-4ee2-a49b-5fc8b573aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.603877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba984fa-e9c6-4ee2-a49b-5fc8b573aa87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.604979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba984fa-e9c6-4ee2-a49b-5fc8b573aa87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.605596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.606324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.606607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.606883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.607581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:05:35.607891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.608737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba984fa-e9c6-4ee2-a49b-5fc8b573aa87 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.609541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.636254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 785/1577] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:05:35.637489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.652175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d2ff2-dfb5-4b02-897e-b60c267e55d1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.658291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.659015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.659298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.659591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] PATH_INFO: `/v3/projects/3ed58deb61194a5ba3f2d46eb7e5680c/users/61401557e7624580b20de6e31c92f785/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.682398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed58deb61194a5ba3f2d46eb7e5680c, user_id=61401557e7624580b20de6e31c92f785, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.684276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:05:35.684276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:05:35.684276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:05:35.688744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.724355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 793/1578] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:35.725959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d155e5f-ace6-4fae-9c09-518c081bd941 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.726325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d155e5f-ace6-4fae-9c09-518c081bd941 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.726591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d155e5f-ace6-4fae-9c09-518c081bd941 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.728180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 794/1579] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 14:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:05:35.729006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d155e5f-ace6-4fae-9c09-518c081bd941 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.758212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:35.785491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 786/1580] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:35 2022] PUT /identity/v3/projects/3ed58deb61194a5ba3f2d46eb7e5680c/users/61401557e7624580b20de6e31c92f785/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:35.786774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:35.820027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d155e5f-ace6-4fae-9c09-518c081bd941 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.886360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d20adf-5a2b-4ff2-af27-765a168f681d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:35.895374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.896385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.896831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.897251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.898455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.901140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:35.905599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:35.906532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:35.908030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:35.908854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:35.909729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:35.909959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:35.912645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.043393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 787/1581] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/projects => generated 375 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.044591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.078188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 795/1582] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:35 2022] POST /identity/v3/users => generated 286 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.079537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.094895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0b7e-b0e8-4ba3-ac65-78614417c525 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.127731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a1df-1d80-490c-951c-26b42685b3d2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.136968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.138097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.138611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.139115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.140273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.145507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.191241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.192318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.192804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.193303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.194684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.198716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.273218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 788/1583] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.274328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.306274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 796/1584] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/projects => generated 405 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.308643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.362932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0bf67-89c3-4db1-a40a-83ba8aa2ec3e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.366416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:36.393026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6479db-7e07-43c9-bd75-1e931c2289f5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.400893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c91da4-19ef-40ea-8395-c8d0aa75377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.401842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c91da4-19ef-40ea-8395-c8d0aa75377d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.402324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c91da4-19ef-40ea-8395-c8d0aa75377d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.402845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c91da4-19ef-40ea-8395-c8d0aa75377d admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.404143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c91da4-19ef-40ea-8395-c8d0aa75377d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.408509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c91da4-19ef-40ea-8395-c8d0aa75377d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.440185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b537f-138b-4078-a200-b4d3a05081a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.440947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b537f-138b-4078-a200-b4d3a05081a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.441273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b537f-138b-4078-a200-b4d3a05081a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.441594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b537f-138b-4078-a200-b4d3a05081a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.442337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b537f-138b-4078-a200-b4d3a05081a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.444851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b537f-138b-4078-a200-b4d3a05081a5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.516955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 797/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/users => generated 296 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.519547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.520160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.520566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.544923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:36.547200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:36.552286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 789/1586] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/projects => generated 357 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.556092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.556583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.557076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.572716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:36.596452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:36.598990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:36.649137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 798/1587] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:36.649137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.702350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 790/1588] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:36.704568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.718244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a653e-28dc-4f02-bc15-85060fdd5190 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.728342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.729514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.730017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.730404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.731283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.733363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.747806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 791/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:36.749109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.754811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718af600-d8ce-4cf6-8d9a-3ca4b128258f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.760844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.761612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.762802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.763188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.763954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.765777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce245-5cd1-4ae5-abcb-747fa1f0d4aa admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.767116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.772436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.773274 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.773649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.774124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.775033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.778009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.905763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 799/1590] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/users => generated 288 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.906782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.917818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 792/1591] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:36.918160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9022-699c-4387-95aa-19690a5ea3d1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.919329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.925072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.925748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.926069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.926347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.927004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.928984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.934306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7f8c-2e38-4017-98e7-2c0922d4ecd8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.941680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.942564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.942930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.943214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.943877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.945518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:36.958765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 793/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:36.959763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:36.971444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5416-67c7-4563-a040-01e2092276b9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:36.976580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:36.977379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:36.977723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:36.978132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:36.978846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:36.980918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.075717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 800/1593] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/users => generated 303 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:37.077578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.135981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 794/1594] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:36 2022] POST /identity/v3/users => generated 279 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:37.136878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.152589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:37.179237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37fe70-d232-4038-906b-6e3c2664ac58 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.184171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.185288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.186038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.186555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.207257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.211119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee74a7-bc29-4b6c-a4c4-11a665304215 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.211295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.213341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.214008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.214292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.214617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.226016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.227575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.234103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 801/1595] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:37.238251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.240657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 795/1596] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:37.241466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.263158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2e69-cdbb-402f-83a9-f125ba1bde29 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.263158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.263158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.263158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.263822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.263822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.263822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.269672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7da03-5667-4840-8cf9-76c6d9d3bcf5 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.277241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 796/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.280727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.281622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.282461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.285250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.286715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.289387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.293523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.296184 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92e1d3-5d81-404c-9107-7e3ce5a83674 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.302768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.303539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.303905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.304252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.304979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.306877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.315065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 802/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.316390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.323945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 797/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.325228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.336171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a69b5-6f3a-4723-ae61-e43132cd9f02 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.342566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.343328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.343678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.344025 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.344239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea4da-5c25-46e1-8811-cc6a231b413a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.345280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.346999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.351903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.352881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.353968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.353968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.356009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.358285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.363111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 803/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.363111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.377022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885d0cc-53c7-4e5a-9186-1f9bd971a850 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.383189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.383821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.384308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.384469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.385061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.386699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.387033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 798/1601] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.390985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.400762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 804/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.401570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.415680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d17a4-400e-4556-98aa-2194123fb57c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.421993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.424247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.424589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.424915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] PATH_INFO: `/v3/projects/c457ca6329364295baa6922524acb58d/users/1dc0cdb78a5f4db3892135af0507668c/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.427140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2afbb-1656-484d-9efb-b07639ad300d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.440728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.443472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.444222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.445076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.447127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.447298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c457ca6329364295baa6922524acb58d, user_id=1dc0cdb78a5f4db3892135af0507668c, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.450904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.451134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.480566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 799/1603] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.483481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.505462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3106-acd4-42af-bbfa-e6874ca9ff8a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.514999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.516123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.517115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.517617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.518869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.522257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.525370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:37.544426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 805/1604] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:37 2022] PUT /identity/v3/projects/c457ca6329364295baa6922524acb58d/users/1dc0cdb78a5f4db3892135af0507668c/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:37.550076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 800/1605] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.551982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.559833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.582328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93ae4c-ae9c-4ac0-a4d7-83377cda9021 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.716602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d6c99-4157-4c02-b025-12f8104e0434 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.720772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.722765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.730073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.730073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.730073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.730623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.730724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] PATH_INFO: `/v3/projects/36049d2090b84443a3bf3bf00558379e/users/7562a0429fdf42948174b517b211884f/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.730827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.730827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.733255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.758765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 801/1606] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.758765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.762064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=36049d2090b84443a3bf3bf00558379e, user_id=7562a0429fdf42948174b517b211884f, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.765417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.773480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aab18e-0af9-4c10-842e-d3df9d019348 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.779508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.780162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.780445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.780721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.781394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.784388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.812178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 802/1607] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:37.812178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:37.831282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c457c-0804-4890-b8e4-641dd4fc8347 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:37.840189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc437f0d-b2a2-4ac8-b6d0-2638e187a42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:37.841379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437f0d-b2a2-4ac8-b6d0-2638e187a42c admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:37.841881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437f0d-b2a2-4ac8-b6d0-2638e187a42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:37.842378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437f0d-b2a2-4ac8-b6d0-2638e187a42c admin admin] PATH_INFO: `/v3/projects/eaba67e3a54f4191b9b924305a83e4eb/users/ff664c7ba095413daad6b3f1ebc49cc0/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:37.870824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437f0d-b2a2-4ac8-b6d0-2638e187a42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaba67e3a54f4191b9b924305a83e4eb, user_id=ff664c7ba095413daad6b3f1ebc49cc0, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:37.877539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437f0d-b2a2-4ac8-b6d0-2638e187a42c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:37.985812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 806/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:37 2022] PUT /identity/v3/projects/36049d2090b84443a3bf3bf00558379e/users/7562a0429fdf42948174b517b211884f/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:37.986587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:38.071620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12f44d-376c-47d0-91e0-e0c969569acb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:38.077733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:38.078759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.079120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.079557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] PATH_INFO: `/v3/projects/f871b0e1d3154b6d8c297b12ae738be0/users/c5b991f5024e4b5dbe8510b7381b1dc8/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.089264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:38.098474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f871b0e1d3154b6d8c297b12ae738be0, user_id=c5b991f5024e4b5dbe8510b7381b1dc8, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:38.102847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:38.152868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 803/1609] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:37 2022] PUT /identity/v3/projects/eaba67e3a54f4191b9b924305a83e4eb/users/ff664c7ba095413daad6b3f1ebc49cc0/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:38.154033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.154545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.155002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.205824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] MFA Rules not processed for user `1dc0cdb78a5f4db3892135af0507668c`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.207622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:38.326324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 804/1610] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:38.326324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:38.355221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 807/1611] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:37 2022] PUT /identity/v3/projects/f871b0e1d3154b6d8c297b12ae738be0/users/c5b991f5024e4b5dbe8510b7381b1dc8/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:38.355221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:38.411204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:38.428431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23150139-0e35-4a2d-9f70-e04c3c7cdaae None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:38.457310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34261c1e-5368-4362-98d6-7906ae223ddc admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:38.461880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efe7b5-1a00-4556-bd9b-52893b6c0b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:38.462574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efe7b5-1a00-4556-bd9b-52893b6c0b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.462856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efe7b5-1a00-4556-bd9b-52893b6c0b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.463132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efe7b5-1a00-4556-bd9b-52893b6c0b0b admin admin] PATH_INFO: `/v3/projects/1420fdd935654e00b1da064f02b5f1ae/users/3d3a5ecdf0a24e079a307b5124b6131b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.479756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efe7b5-1a00-4556-bd9b-52893b6c0b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1420fdd935654e00b1da064f02b5f1ae, user_id=3d3a5ecdf0a24e079a307b5124b6131b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:38.482492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efe7b5-1a00-4556-bd9b-52893b6c0b0b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:38.504954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:38.505666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.505995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.506277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] PATH_INFO: `/v3/projects/596c91c754774c1692c937c68a2756c5/users/242d325a94a5478090aa9388851ffdb1/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.526556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=596c91c754774c1692c937c68a2756c5, user_id=242d325a94a5478090aa9388851ffdb1, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:38.529786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:38.585541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 808/1612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:38 2022] PUT /identity/v3/projects/1420fdd935654e00b1da064f02b5f1ae/users/3d3a5ecdf0a24e079a307b5124b6131b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:38.587130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a1706-c36f-4967-a387-00f1c61550cd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.587584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a1706-c36f-4967-a387-00f1c61550cd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.587864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a1706-c36f-4967-a387-00f1c61550cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.614976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e33a1706-c36f-4967-a387-00f1c61550cd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.614976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e33a1706-c36f-4967-a387-00f1c61550cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:38.687153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 809/1613] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:38.689180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564ef19-c768-49c6-97d5-8085779f465c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.689473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564ef19-c768-49c6-97d5-8085779f465c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.689754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564ef19-c768-49c6-97d5-8085779f465c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.693552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 805/1614] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:38 2022] PUT /identity/v3/projects/596c91c754774c1692c937c68a2756c5/users/242d325a94a5478090aa9388851ffdb1/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:38.694560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:38.711159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8564ef19-c768-49c6-97d5-8085779f465c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.712838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8564ef19-c768-49c6-97d5-8085779f465c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:38.773431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2319e-ca0f-4035-b9c0-423a0982a8db admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:38.784769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749a83d-e2d3-4c7e-88f6-a388dda8a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:38.784769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749a83d-e2d3-4c7e-88f6-a388dda8a83a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.784769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749a83d-e2d3-4c7e-88f6-a388dda8a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.784769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749a83d-e2d3-4c7e-88f6-a388dda8a83a admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.785301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749a83d-e2d3-4c7e-88f6-a388dda8a83a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:38.786288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749a83d-e2d3-4c7e-88f6-a388dda8a83a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:38.793929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 810/1615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:38.798919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192593e7-668b-4f9f-9534-a719641a466d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.798919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192593e7-668b-4f9f-9534-a719641a466d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.798919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192593e7-668b-4f9f-9534-a719641a466d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.818470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-192593e7-668b-4f9f-9534-a719641a466d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.819735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-192593e7-668b-4f9f-9534-a719641a466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:38.828046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 811/1616] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:38.829341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.829627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.829951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.838329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:38.859148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] MFA Rules not processed for user `3d3a5ecdf0a24e079a307b5124b6131b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.861433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:38.939423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 806/1617] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/projects => generated 371 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:38.941627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.942134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.942422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.952272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 812/1618] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:38.953295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:38.988234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.989359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:39.063814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b2662-5596-4fa6-b950-33a81949ed06 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.087661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 807/1619] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:39.088667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:39.101551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:39.155309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.155987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.156280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.156560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.169488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:39.171706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:39.197375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 813/1620] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:39.198417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:39.204830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:39.205680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d9436-a4d1-4023-a73c-b9acd7fed0e6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.213023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.214426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.215052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.215667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.234320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f68b4-7b45-4e47-885d-8340ea6c16b9 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.235830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:39.239249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.239888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.240068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:39.240267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.240434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.257982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:39.260751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:39.263443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 808/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:39.265283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:39.287218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 814/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:39.288319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378124eb-b2a7-4e0e-942c-0406102b51f7 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.290109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:39.306046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.307764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.308749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.310771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.312635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:39.315065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7c5c-d189-4d47-9fc5-2e1630387b48 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.317455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:39.324253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.325971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.327332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.327906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.330201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:39.337494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:39.511506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 815/1623] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:39 2022] POST /identity/v3/projects => generated 355 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:39.511506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:39.512550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 809/1624] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:39 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:39.513864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:39.545095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b3eca-94f8-47a6-bc13-8f83539046d8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.580199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:39.709389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d6cb9-96ee-4521-a948-59587d94fec1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:39.715638 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a132f0e-97ba-4415-92b0-735a973913ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.717056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a132f0e-97ba-4415-92b0-735a973913ad service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.717608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a132f0e-97ba-4415-92b0-735a973913ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.718082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a132f0e-97ba-4415-92b0-735a973913ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.746118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:39.746743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:39.750472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:39.751341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:39.753929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:39.757231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:39.819056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 810/1625] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:39.824420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.126721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a132f0e-97ba-4415-92b0-735a973913ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.126721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a132f0e-97ba-4415-92b0-735a973913ad service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.126721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 816/1626] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:40.128386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7ec09-4f90-443c-8117-ea55f2eb4c76 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.129645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b0d45-1a9a-496c-8359-4e2988d851f4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.129645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b0d45-1a9a-496c-8359-4e2988d851f4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.129645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b0d45-1a9a-496c-8359-4e2988d851f4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.129645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 817/1627] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 14:05:39 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:05:40.129645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b0d45-1a9a-496c-8359-4e2988d851f4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.133712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.133712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.133712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.133712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.135931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.137424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b0d45-1a9a-496c-8359-4e2988d851f4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.137424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ecd5-dd33-47e1-9627-3346fb2efb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.137424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ecd5-dd33-47e1-9627-3346fb2efb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.137424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ecd5-dd33-47e1-9627-3346fb2efb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.139291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ecd5-dd33-47e1-9627-3346fb2efb77 admin admin] PATH_INFO: `/v3/projects/59fc2b9959c04130838f48189657249f/users/6f5acd2942d340b898873e536dc55604/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.140439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.140907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ecd5-dd33-47e1-9627-3346fb2efb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fc2b9959c04130838f48189657249f, user_id=6f5acd2942d340b898873e536dc55604, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.140907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ecd5-dd33-47e1-9627-3346fb2efb77 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.140907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 818/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:39 2022] PUT /identity/v3/projects/59fc2b9959c04130838f48189657249f/users/6f5acd2942d340b898873e536dc55604/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:40.140907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.140907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.140907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.194231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 811/1629] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:39 2022] POST /identity/v3/users => generated 284 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:40.194231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.213093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:40.216801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:40.223874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a1093-0c91-4297-96a6-a70fcb2f2106 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.332086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.333379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.334127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.334606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.335710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.340997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.374264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 812/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:40.374928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.387635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:40.398606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea077c-1402-4ebf-ad68-3b9b2679786e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.404703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd22ed-dd85-435b-96d2-972d398fef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.405520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd22ed-dd85-435b-96d2-972d398fef25 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.406255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd22ed-dd85-435b-96d2-972d398fef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.406984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd22ed-dd85-435b-96d2-972d398fef25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.408175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd22ed-dd85-435b-96d2-972d398fef25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.410159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd22ed-dd85-435b-96d2-972d398fef25 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.414112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 819/1631] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:40.418177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.430964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 813/1632] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:40.433665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20497e0-d3c1-494f-a0c2-3f0d7a421dd6 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.434603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20497e0-d3c1-494f-a0c2-3f0d7a421dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.435151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20497e0-d3c1-494f-a0c2-3f0d7a421dd6 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.437875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 814/1633] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 14:05:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:05:40.439755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20497e0-d3c1-494f-a0c2-3f0d7a421dd6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.458524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20497e0-d3c1-494f-a0c2-3f0d7a421dd6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.463926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.464960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.465251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.467530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d9f0e-06b4-41c4-b379-4ccbcc6af233 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.467658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] PATH_INFO: `/v3/projects/2dbde8c1a1da4245a21f7b70af70af82/users/abf8dfa5e1a2481fb6cb3c5a2e47b7d0/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.471650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c91346-6fd9-476a-9eb9-537bb8d6186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.472449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c91346-6fd9-476a-9eb9-537bb8d6186d admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.472848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c91346-6fd9-476a-9eb9-537bb8d6186d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.474021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c91346-6fd9-476a-9eb9-537bb8d6186d admin admin] PATH_INFO: `/v3/projects/59fc2b9959c04130838f48189657249f/users/6f5acd2942d340b898873e536dc55604/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.476545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c91346-6fd9-476a-9eb9-537bb8d6186d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fc2b9959c04130838f48189657249f, user_id=6f5acd2942d340b898873e536dc55604, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.479901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c91346-6fd9-476a-9eb9-537bb8d6186d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.486544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbde8c1a1da4245a21f7b70af70af82, user_id=abf8dfa5e1a2481fb6cb3c5a2e47b7d0, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.489497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.502331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 820/1634] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:40 2022] PUT /identity/v3/projects/59fc2b9959c04130838f48189657249f/users/6f5acd2942d340b898873e536dc55604/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:40.503902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f3cd9-df73-4382-9f90-6eeb879c8da2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.512144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f3cd9-df73-4382-9f90-6eeb879c8da2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.512144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f3cd9-df73-4382-9f90-6eeb879c8da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.538491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a46f3cd9-df73-4382-9f90-6eeb879c8da2 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:40.540163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a46f3cd9-df73-4382-9f90-6eeb879c8da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:40.667605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 821/1635] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:40.674586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.675465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.676142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.729978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 815/1636] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:40 2022] PUT /identity/v3/projects/2dbde8c1a1da4245a21f7b70af70af82/users/abf8dfa5e1a2481fb6cb3c5a2e47b7d0/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:40.731604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.770179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] MFA Rules not processed for user `6f5acd2942d340b898873e536dc55604`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:40.770179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:40.842983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:40.845274 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966c0e2-1d75-434f-8f6c-fc10c6562c60 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:40.849667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:40.850462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.850751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.851231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.945351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:40.947550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:40.964908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 816/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:40.965831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:40.995048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 822/1638] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:41.004161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:41.033368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac0e38-063d-4f58-9d97-eb990027554f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:41.048206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:41.049585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:41.049585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:41.049585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:41.072139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:41.073809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:41.093595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5cb97-e9ee-4a5b-b083-ea7b65227900 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:41.100577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 823/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:05:41.102105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2164032-2131-4276-8e60-016177b5a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:41.103138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2164032-2131-4276-8e60-016177b5a1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:41.103743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2164032-2131-4276-8e60-016177b5a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:41.104288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2164032-2131-4276-8e60-016177b5a1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:41.105222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2164032-2131-4276-8e60-016177b5a1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:41.107499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2164032-2131-4276-8e60-016177b5a1ad admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:41.130035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 817/1640] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:41.142120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:41.177534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85d6a2-ddbe-4ba7-be36-39adf7536c14 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:41.202443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:41.203302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:41.203833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:41.204220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] PATH_INFO: `/v3/projects/2dbde8c1a1da4245a21f7b70af70af82/users/abf8dfa5e1a2481fb6cb3c5a2e47b7d0/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:41.213612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbde8c1a1da4245a21f7b70af70af82, user_id=abf8dfa5e1a2481fb6cb3c5a2e47b7d0, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:41.224467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:41.262692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 824/1641] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:41 2022] PUT /identity/v3/projects/2dbde8c1a1da4245a21f7b70af70af82/users/abf8dfa5e1a2481fb6cb3c5a2e47b7d0/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:41.268048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:41.268593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:41.269186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:41.326363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] MFA Rules not processed for user `abf8dfa5e1a2481fb6cb3c5a2e47b7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:41.329403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:41.589924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 818/1642] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:41.775005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:41.871022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f82a-535f-4dfc-9bf6-ab6cf1491e98 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:41.876308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:41.877389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:41.877389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:41.877651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:41.890277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:41.892390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:41.904047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 825/1643] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:05:42.060101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:42.103395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a0cab-1dd5-489d-9588-4c3e2f84797e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:42.112806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fbd37-e0f1-4d43-9d54-d555b182c523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:42.120537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fbd37-e0f1-4d43-9d54-d555b182c523 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:42.120537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fbd37-e0f1-4d43-9d54-d555b182c523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:42.120537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fbd37-e0f1-4d43-9d54-d555b182c523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:42.236072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fbd37-e0f1-4d43-9d54-d555b182c523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:42.237049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fbd37-e0f1-4d43-9d54-d555b182c523 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:42.255897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 819/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:42.276379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:42.487728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:42.535383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba5f8a-4c9d-4f64-ab80-635017020d2f service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:42.542294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:42.543402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:42.543912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:42.544416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:42.671133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:42.675843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:42.690953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 826/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:42.700961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:42.704382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:42.704382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:42.770353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] MFA Rules not processed for user `242d325a94a5478090aa9388851ffdb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:42.773045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:42.791420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:42.934694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 820/1646] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:42.940010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:43.135316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70954111-eeb7-480d-aa6c-6fc52fc41b8d service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:43.158900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:43.160048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:43.160807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:43.161301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:43.162836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:43.166123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:43.284848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 827/1647] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:42 2022] POST /identity/v3/projects => generated 403 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:43.290828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:43.308839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd34f93-fead-4fa3-8e1e-0082b3f4e63b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:43.459196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:43.460265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:43.460743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:43.461220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:43.462599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:43.490544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:43.702246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 821/1648] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:43 2022] POST /identity/v3/users => generated 308 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:43.702392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:43.709010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785501-8faf-4891-9f14-5b6cf3aee26d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:43.717157 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:43.717498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:43.717786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:43.718106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:43.718827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:43.721000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:43.739567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 828/1649] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:43.740663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:43.759316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834ff6-1c57-4a23-8b69-72e4b7ca2228 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:43.759316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:43.759316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:43.759316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:43.759806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] PATH_INFO: `/v3/projects/615f4fbb44e642d096e847596b29e315/users/072d8e6888474322b176c44ab04d7085/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:43.779989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615f4fbb44e642d096e847596b29e315, user_id=072d8e6888474322b176c44ab04d7085, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:43.783225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:44.086004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 822/1650] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:43 2022] PUT /identity/v3/projects/615f4fbb44e642d096e847596b29e315/users/072d8e6888474322b176c44ab04d7085/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:05:44.086094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:44.102049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b4d3f-7bcf-4a9c-8c00-ff6c3d47b700 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:44.253412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d604fa-11b9-454e-99ed-271cb0de80ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:44.254865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d604fa-11b9-454e-99ed-271cb0de80ed admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:44.255636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d604fa-11b9-454e-99ed-271cb0de80ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:44.256686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d604fa-11b9-454e-99ed-271cb0de80ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:44.257747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d604fa-11b9-454e-99ed-271cb0de80ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:44.260820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d604fa-11b9-454e-99ed-271cb0de80ed admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:44.292776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:44.313255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 829/1651] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:44.321476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce50b29-18d8-4ac3-abf1-8df6479fba22 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:44.328749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:44.329508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:44.329796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:44.330503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] PATH_INFO: `/v3/projects/615f4fbb44e642d096e847596b29e315/users/072d8e6888474322b176c44ab04d7085/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:44.335231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:44.342080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] RBAC: Authorizing `identity:create_grant(project_id=615f4fbb44e642d096e847596b29e315, user_id=072d8e6888474322b176c44ab04d7085, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:44.343604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:44.368443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 823/1652] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:44 2022] PUT /identity/v3/projects/615f4fbb44e642d096e847596b29e315/users/072d8e6888474322b176c44ab04d7085/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:51.473604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd0d7-5f56-4df3-bfc7-1b874aa5d3fc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:51.474298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd0d7-5f56-4df3-bfc7-1b874aa5d3fc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:51.474371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd0d7-5f56-4df3-bfc7-1b874aa5d3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:51.638964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-af3dd0d7-5f56-4df3-bfc7-1b874aa5d3fc None None] MFA Rules not processed for user `072d8e6888474322b176c44ab04d7085`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:51.647391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-af3dd0d7-5f56-4df3-bfc7-1b874aa5d3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:51.718371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:51.983712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 830/1653] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:51 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 512 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:53.396280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:53.629146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a75f9c-b2a3-4bf2-9e67-c6968b93885b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:53.652668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:53.655816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:53.656188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:53.656420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] PATH_INFO: `/v3/users/d58fb7cea93a4b46ada7e58657768096` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:53.667775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58fb7cea93a4b46ada7e58657768096)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:53.671947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: WARNING py.warnings [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:05:53.671947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: warnings.warn(msg) Sep 01 14:05:53.671947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: Sep 01 14:05:53.676133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:54.196982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58fb7cea93a4b46ada7e58657768096'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:05:54.334275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58fb7cea93a4b46ada7e58657768096'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:05:54.398167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58fb7cea93a4b46ada7e58657768096'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:05:54.401329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58fb7cea93a4b46ada7e58657768096'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:05:54.406440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:54.437572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3654162b-741c-4c83-8d0f-d3c7bf491724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58fb7cea93a4b46ada7e58657768096'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:05:54.459794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 824/1654] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:05:53 2022] DELETE /identity/v3/users/d58fb7cea93a4b46ada7e58657768096 => generated 0 bytes in 1064 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:54.465540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:54.467309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:54.468656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:54.511514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91115470-cd1d-4dfa-b0ac-6536f5c9c000 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:54.512316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91115470-cd1d-4dfa-b0ac-6536f5c9c000 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:54.512795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91115470-cd1d-4dfa-b0ac-6536f5c9c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:54.519702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:54.522376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:54.557385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-91115470-cd1d-4dfa-b0ac-6536f5c9c000 None None] MFA Rules not processed for user `c5b991f5024e4b5dbe8510b7381b1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:54.559420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-91115470-cd1d-4dfa-b0ac-6536f5c9c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:54.685960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 825/1655] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:05:54.696784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:54.697003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:54.697205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:54.720131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] MFA Rules not processed for user `c5b991f5024e4b5dbe8510b7381b1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:54.721564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:54.733371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 831/1656] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:54.734981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 826/1657] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:05:54.764793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:54.765933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:54.769018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:54.771007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:54.913239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6ac0d-4108-4823-b11e-309b43c2be47 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:54.931370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793df7-727a-4a6f-b232-5429a6dfff3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:54.936199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793df7-727a-4a6f-b232-5429a6dfff3b service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:54.936199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793df7-727a-4a6f-b232-5429a6dfff3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:54.936199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793df7-727a-4a6f-b232-5429a6dfff3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:54.970070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8793df7-727a-4a6f-b232-5429a6dfff3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:54.972109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8793df7-727a-4a6f-b232-5429a6dfff3b service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:55.001617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 827/1658] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:05:55.020324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90395b-605c-4e98-9e5a-8bc81be46f0b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:55.031147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e65f49-d1a2-43d1-982b-b75509a00f24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:55.031147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e65f49-d1a2-43d1-982b-b75509a00f24 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:55.031147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e65f49-d1a2-43d1-982b-b75509a00f24 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:55.031147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e65f49-d1a2-43d1-982b-b75509a00f24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:55.047911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e65f49-d1a2-43d1-982b-b75509a00f24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:55.050100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e65f49-d1a2-43d1-982b-b75509a00f24 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:55.068072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 832/1659] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:56.153626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd432db-3ef3-42aa-b96b-1a687a19bb32 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:56.154389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd432db-3ef3-42aa-b96b-1a687a19bb32 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:56.154389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd432db-3ef3-42aa-b96b-1a687a19bb32 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:56.157044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 828/1660] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:05:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:05:56.186398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5028ee-b0a8-408b-8908-19b032d1d2a0 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:56.187269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5028ee-b0a8-408b-8908-19b032d1d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:56.187410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5028ee-b0a8-408b-8908-19b032d1d2a0 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:56.190642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 833/1661] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 14:05:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:05:56.223702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a01b0-6f57-4110-bcec-f743e5d1ec5d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:56.223702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a01b0-6f57-4110-bcec-f743e5d1ec5d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:56.223702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a01b0-6f57-4110-bcec-f743e5d1ec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:56.393527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d34a01b0-6f57-4110-bcec-f743e5d1ec5d None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:56.395812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d34a01b0-6f57-4110-bcec-f743e5d1ec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:56.457816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 829/1662] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:05:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 01 14:05:56.511864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5028ee-b0a8-408b-8908-19b032d1d2a0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:56.523581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2f5028ee-b0a8-408b-8908-19b032d1d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:56.579985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5028ee-b0a8-408b-8908-19b032d1d2a0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:56.580312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:56.580769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:56.581437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:56.596873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:56.598368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:56.601536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:56.602085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:56.624173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:56.632931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:56.655240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 834/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:56.691836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:56.693661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:57.011653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 830/1664] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:05:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:57.302094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:57.303902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:57.340081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16a384-25e0-4f40-b899-0af994cc8686 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:57.362739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e849f7a-1e8b-4c47-9669-37586734f3b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:57.363261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e849f7a-1e8b-4c47-9669-37586734f3b6 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:57.363551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e849f7a-1e8b-4c47-9669-37586734f3b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:57.363836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e849f7a-1e8b-4c47-9669-37586734f3b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:57.390028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e849f7a-1e8b-4c47-9669-37586734f3b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:57.391694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e849f7a-1e8b-4c47-9669-37586734f3b6 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:57.418824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 835/1665] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:57.540937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:58.706655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:58.764748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3cf7-4ff1-4544-930f-516eb86f7c16 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:58.794213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:58.800413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:58.801789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:58.802521 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:58.816547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:58.856472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:58.857415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:58.860014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:58.887007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:58.893169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:58.919763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:58.934818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 831/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:58.976455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] MFA Rules not processed for user `61401557e7624580b20de6e31c92f785`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:58.976455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:59.294996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 836/1667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:05:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:59.330690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:59.383673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6ac4e-77c3-491b-9ed9-ba3896e1a60a service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:59.418391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5926b3-f7f8-4479-afc8-42df748fcdf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:59.422791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5926b3-f7f8-4479-afc8-42df748fcdf8 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:59.423620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5926b3-f7f8-4479-afc8-42df748fcdf8 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:59.424215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5926b3-f7f8-4479-afc8-42df748fcdf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:59.468304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5926b3-f7f8-4479-afc8-42df748fcdf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:59.474515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:59.478440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:59.488265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5926b3-f7f8-4479-afc8-42df748fcdf8 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:59.582198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 832/1668] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:59.607951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4b49-581d-485e-884f-17fdf64eab55 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:59.666155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:59.667299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:59.667767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:59.668277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] PATH_INFO: `/v3/projects/238327a3ec7946ada50cc2a71d542415` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:59.692612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238327a3ec7946ada50cc2a71d542415)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:59.694251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:05:59.694251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Sep 01 14:05:59.694251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: Sep 01 14:05:59.713941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:00.073164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238327a3ec7946ada50cc2a71d542415', 'request_id': 'req-039c00cc-4c13-4026-9cce-f3c2c89bde17'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:00.090204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238327a3ec7946ada50cc2a71d542415', 'request_id': 'req-039c00cc-4c13-4026-9cce-f3c2c89bde17'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:00.101653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238327a3ec7946ada50cc2a71d542415', 'request_id': 'req-039c00cc-4c13-4026-9cce-f3c2c89bde17'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:00.276345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] The token cache is being invalidate because project 238327a3ec7946ada50cc2a71d542415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:00.276345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:00.276345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-039c00cc-4c13-4026-9cce-f3c2c89bde17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:00.282501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 837/1669] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:05:59 2022] DELETE /identity/v3/projects/238327a3ec7946ada50cc2a71d542415 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:00.284109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:00.284636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:00.284878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:00.351902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:00.353445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:00.579352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 833/1670] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:00.581735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:00.582211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:00.582611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:00.653145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:00.658683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:00.683235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 838/1671] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:00.687382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:00.705362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69b69-044d-47f1-bf82-36ede5036d29 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:00.721334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:00.722183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:00.722572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:00.722957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:00.725563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:00.726076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:00.728045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:00.797358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 834/1672] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:00.798359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:00.825997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230198b-11ce-4078-8066-b790b3b14e8c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:00.864296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b8bd-1302-4027-b6a1-2aba8233c27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:00.865541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b8bd-1302-4027-b6a1-2aba8233c27b admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:00.866008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b8bd-1302-4027-b6a1-2aba8233c27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:00.866466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b8bd-1302-4027-b6a1-2aba8233c27b admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:00.868032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b8bd-1302-4027-b6a1-2aba8233c27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:00.871083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b8bd-1302-4027-b6a1-2aba8233c27b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:01.170039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:01.179851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 839/1673] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:00 2022] POST /identity/v3/projects => generated 363 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:01.191046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cb3cc-f0dc-4910-95e8-bb1fd1b63d56 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:01.259478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:01.264050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:01.264050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:01.347108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] MFA Rules not processed for user `ff664c7ba095413daad6b3f1ebc49cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:01.363572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:01.412321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:01.416557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:01.416557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:01.416557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:01.416557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:01.453421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:01.556651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:01.646431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 835/1674] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:01 2022] POST /identity/v3/users => generated 282 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:01.652296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:01.662584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 840/1675] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:01.682717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:01.686394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:01.687904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1519-ef11-4de8-862f-c4dfdc5574ea admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:01.707059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:01.725508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:01.726037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:01.729216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:01.729614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:01.734189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:01.765286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 836/1676] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:01.767796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:01.802945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a0d60-7988-44a8-84ae-19821b21fd13 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:01.819500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:01.820291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:01.821146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:01.821457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:01.822238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:01.824036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:01.859978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 837/1677] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:01.867227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:01.905070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6a9dd-905f-45ed-a0c2-8c9e82407aff admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:01.942647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ded2c-c546-4697-9b10-1cf9427c3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:01.949080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ded2c-c546-4697-9b10-1cf9427c3995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:01.949433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ded2c-c546-4697-9b10-1cf9427c3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:01.950689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ded2c-c546-4697-9b10-1cf9427c3995 admin admin] PATH_INFO: `/v3/projects/16b26a6f01844176b377642e6028d667/users/8b1b6818b654453e90c2cdfd5cb04d4a/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:01.994756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ded2c-c546-4697-9b10-1cf9427c3995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b26a6f01844176b377642e6028d667, user_id=8b1b6818b654453e90c2cdfd5cb04d4a, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:01.998126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ded2c-c546-4697-9b10-1cf9427c3995 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:02.010375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:02.086556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8ccd3-8b1f-4765-af16-461d1e171d9f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:02.100514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:02.101438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:02.101951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:02.102391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:02.118007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 838/1678] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:01 2022] PUT /identity/v3/projects/16b26a6f01844176b377642e6028d667/users/8b1b6818b654453e90c2cdfd5cb04d4a/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:02.136535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:02.136849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:02.137326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:02.203225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:02.205028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:02.264640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 841/1679] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:02.265690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:02.275235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:02.352171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] MFA Rules not processed for user `8b1b6818b654453e90c2cdfd5cb04d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:02.366061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de937-f928-4de6-9ae0-770d11a56d0a service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:02.367353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:02.686581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:02.692659 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:02.694118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:02.694656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] PATH_INFO: `/v3/users/242d325a94a5478090aa9388851ffdb1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:02.697771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242d325a94a5478090aa9388851ffdb1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:02.701068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:02.856721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 839/1680] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 721 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:02.877233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:02.915826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3c0e-186b-4caf-9b50-8c019ce76635 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:02.932082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:02.933019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:02.933451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:02.939622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:02.939622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:02.939622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:03.107100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 840/1681] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:02 2022] POST /identity/v3/projects => generated 365 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:03.113887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:03.129867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242d325a94a5478090aa9388851ffdb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:03.141395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242d325a94a5478090aa9388851ffdb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:03.242582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242d325a94a5478090aa9388851ffdb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:03.337049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242d325a94a5478090aa9388851ffdb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:03.372212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242d325a94a5478090aa9388851ffdb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:03.414510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 842/1682] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:02 2022] DELETE /identity/v3/users/242d325a94a5478090aa9388851ffdb1 => generated 0 bytes in 1141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:03.434821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3df91-99fa-4b13-ad64-b491e85a77c8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:03.745581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:03.746748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:03.747266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:03.748931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:03.750791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:03.760306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:03.885213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:03.897446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 841/1683] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:03 2022] POST /identity/v3/users => generated 289 bytes in 784 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:03.902841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:03.905301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:03.970642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:04.005164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974026-af4f-4a05-a0d2-7dfe5f777b77 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:04.044618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:04.046241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:04.046982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:04.048429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:04.051486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:04.060872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:04.103264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 843/1684] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:04.109693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:04.133669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be253d26-e0b6-4eaa-bb00-d0b8fd6c76b6 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:04.162818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:04.165446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:04.167701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:04.168192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] PATH_INFO: `/v3/projects/60a7fdd5497143a9b2484956115dc1fa/users/b797a0d684494aa691ba73dd4388bd69/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:04.214399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a7fdd5497143a9b2484956115dc1fa, user_id=b797a0d684494aa691ba73dd4388bd69, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:04.220885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:04.364886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 842/1685] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:04 2022] PUT /identity/v3/projects/60a7fdd5497143a9b2484956115dc1fa/users/b797a0d684494aa691ba73dd4388bd69/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:04.371337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:04.387826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453eb55a-4656-4497-9802-c47b26d30749 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:04.530759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3c509-bb41-4181-bb47-b3eb9d981d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:04.531438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3c509-bb41-4181-bb47-b3eb9d981d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:04.531946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3c509-bb41-4181-bb47-b3eb9d981d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:04.532085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3c509-bb41-4181-bb47-b3eb9d981d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:04.533439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3c509-bb41-4181-bb47-b3eb9d981d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:04.538586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3c509-bb41-4181-bb47-b3eb9d981d03 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:04.569854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 844/1686] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:04.584122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:04.600236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964eb8a-0db8-4732-bfec-c7b1021cac82 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:04.620464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:04.621183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:04.621463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:04.621985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] PATH_INFO: `/v3/projects/60a7fdd5497143a9b2484956115dc1fa/users/b797a0d684494aa691ba73dd4388bd69/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:04.628743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a7fdd5497143a9b2484956115dc1fa, user_id=b797a0d684494aa691ba73dd4388bd69, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:04.631848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:04.663528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 843/1687] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:04 2022] PUT /identity/v3/projects/60a7fdd5497143a9b2484956115dc1fa/users/b797a0d684494aa691ba73dd4388bd69/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:04.683768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc174-96cf-42cf-9896-1a1254dba323 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:04.684039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc174-96cf-42cf-9896-1a1254dba323 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:04.684504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc174-96cf-42cf-9896-1a1254dba323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:04.764350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d5adc174-96cf-42cf-9896-1a1254dba323 None None] MFA Rules not processed for user `b797a0d684494aa691ba73dd4388bd69`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:04.773365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d5adc174-96cf-42cf-9896-1a1254dba323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:05.144547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 845/1688] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:05.201891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:05.204496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:05.305155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0706-23a3-4e62-a77d-8a3f24ae4ee5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:05.305663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0706-23a3-4e62-a77d-8a3f24ae4ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:05.306208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0706-23a3-4e62-a77d-8a3f24ae4ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:05.380567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05392-de68-4185-a0b1-a6172e9fe5e2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:05.418218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:05.424332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:05.428414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:05.429925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:05.470489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:05.474882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:05.489074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4ccc0706-23a3-4e62-a77d-8a3f24ae4ee5 None None] MFA Rules not processed for user `7562a0429fdf42948174b517b211884f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:05.508574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4ccc0706-23a3-4e62-a77d-8a3f24ae4ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:05.530628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 844/1689] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:05.539335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:05.545650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:05.736737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:05.803160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 846/1690] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:05.820177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:05.820866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:05.822018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:05.856629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c6c9-6d6d-4600-943b-af2cb4435582 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:05.862883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:05.863525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:05.863876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:05.864274 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] PATH_INFO: `/v3/projects/596c91c754774c1692c937c68a2756c5` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:05.868513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=596c91c754774c1692c937c68a2756c5)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:05.878937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:05.908369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] MFA Rules not processed for user `7562a0429fdf42948174b517b211884f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:05.910498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:05.980673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 847/1691] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:05.997592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:06.003921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:06.226041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596c91c754774c1692c937c68a2756c5', 'request_id': 'req-6c5bec50-5093-47bd-a657-01ab58d0b0c5'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:06.278155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596c91c754774c1692c937c68a2756c5', 'request_id': 'req-6c5bec50-5093-47bd-a657-01ab58d0b0c5'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:06.397230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596c91c754774c1692c937c68a2756c5', 'request_id': 'req-6c5bec50-5093-47bd-a657-01ab58d0b0c5'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:06.428305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:06.436835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567b96a-6578-4a25-8761-640d922c56c6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:06.487562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] The token cache is being invalidate because project 596c91c754774c1692c937c68a2756c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:06.492417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:06.495863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:06.514661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 845/1692] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:05 2022] DELETE /identity/v3/projects/596c91c754774c1692c937c68a2756c5 => generated 0 bytes in 971 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:06.521298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:06.531228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:06.794891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:06.794891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:06.795676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:06.795676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:06.798417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:06.909454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bec50-5093-47bd-a657-01ab58d0b0c5 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:06.947519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:06.951205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:06.951510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:06.951796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] PATH_INFO: `/v3/users/072d8e6888474322b176c44ab04d7085` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:06.960546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] RBAC: Authorizing `identity:delete_user(user_id=072d8e6888474322b176c44ab04d7085)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:06.975386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:06.987841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:06.999964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4efe8d-53f7-41c9-9e1a-66312fd3841c service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:07.040530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 848/1693] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 1043 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:07.044811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:07.045127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:07.045381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:07.123496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:07.239656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:07.247700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:07.347504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072d8e6888474322b176c44ab04d7085'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:07.511568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072d8e6888474322b176c44ab04d7085'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:07.561434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072d8e6888474322b176c44ab04d7085'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:07.562430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072d8e6888474322b176c44ab04d7085'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:07.574299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d1a6ff65-f5e1-4462-aab9-a0dea3b0074b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072d8e6888474322b176c44ab04d7085'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:07.586618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 846/1694] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:06 2022] DELETE /identity/v3/users/072d8e6888474322b176c44ab04d7085 => generated 0 bytes in 1071 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:07.589130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:07.589426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:07.589889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:07.628889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 849/1695] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:06:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:07.633312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:07.658850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:07.693939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] MFA Rules not processed for user `61401557e7624580b20de6e31c92f785`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:07.699176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:07.919126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91bdc9-8bb8-46f9-a6dd-9624e2b561e3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:08.039609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 847/1696] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:08.171744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:08.172424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:08.172803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:08.173391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:08.187156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:08.362511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:08.364717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c4d40-0942-49d1-858e-732f3bb0a86a service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:08.371677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:08.397144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 850/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:08.407247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:08.415470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:08.520856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:08.839488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860bd9-3766-4344-b640-a3f615c4eeb1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:08.846620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:08.847404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:08.847548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:08.847830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:08.850073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:08.909104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d432d-cbfb-49c7-a549-c6f0e59187f6 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:08.909506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d432d-cbfb-49c7-a549-c6f0e59187f6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:08.909754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d432d-cbfb-49c7-a549-c6f0e59187f6 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:08.911874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 851/1698] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 14:06:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:06:08.938407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:08.939926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:08.939926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:09.033647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:09.035517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:09.059352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:09.063285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:09.098923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 848/1699] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:06:09.201239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:09.206798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:09.422021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 852/1700] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:06:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 01 14:06:09.443264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:09.446652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:09.509922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e5f55-bfd5-4fe5-b8c2-96ff5c90cb97 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:09.537858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69122-0cdc-4a1c-b72f-fc92970bdfaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:09.542342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69122-0cdc-4a1c-b72f-fc92970bdfaa service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:09.544441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69122-0cdc-4a1c-b72f-fc92970bdfaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:09.545197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69122-0cdc-4a1c-b72f-fc92970bdfaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:09.608499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69122-0cdc-4a1c-b72f-fc92970bdfaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:09.624081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69122-0cdc-4a1c-b72f-fc92970bdfaa service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:09.697048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5a19-d71f-47f4-bcc5-b3f4013619c5 service swift] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:09.718745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 853/1701] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:09.735065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:09.745018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:09.746038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:09.748053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] PATH_INFO: `/v3/projects/615f4fbb44e642d096e847596b29e315` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:09.751244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615f4fbb44e642d096e847596b29e315)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:09.760045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:10.077328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615f4fbb44e642d096e847596b29e315', 'request_id': 'req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:10.090295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:10.125006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615f4fbb44e642d096e847596b29e315', 'request_id': 'req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:10.278268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615f4fbb44e642d096e847596b29e315', 'request_id': 'req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:10.367304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] The token cache is being invalidate because project 615f4fbb44e642d096e847596b29e315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:10.373618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:10.377568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-baa081ed-7b77-45dd-8fbb-f5f1bffd7bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:10.400765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 849/1702] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:09 2022] DELETE /identity/v3/projects/615f4fbb44e642d096e847596b29e315 => generated 0 bytes in 1197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:10.403221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:10.403756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:10.403901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:10.506836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:10.510874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:10.964168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 854/1703] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:10.974134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:10.974970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:10.974970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:11.055592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:11.059593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:11.100656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 850/1704] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:11.102695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:11.156442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19db99a-97ba-4dd4-8cee-99487f76c68e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:11.213030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:11.214833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:11.215580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:11.234574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:11.235593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:11.236195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:11.245623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c28a2-cb45-4c7f-9471-c3cec3e5cd1f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:11.334595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 855/1705] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:11.337128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:11.357297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94568339-82f2-4af5-9d35-143b81f94449 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:11.394548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:11.398629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:11.399747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:11.404161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:11.405270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:11.414451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:11.544569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:11.545134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:11.545604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:11.621826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 851/1706] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:11 2022] POST /identity/v3/projects => generated 373 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:11.625227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:11.663690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:11.666945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:11.888860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22b67-47bb-4577-b550-eba8a82d88b6 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:11.906583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:11.911960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 856/1707] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Sep 1 14:06:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:11.916760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:11.917059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:11.918400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:11.918400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:11.922401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:11.936619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:11.940739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:12.008484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecabb23-3a94-406a-8133-bbaa6cb6eca9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:12.237586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:12.237586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:12.237586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:12.237586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:12.261169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:12.261169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:12.261169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 857/1708] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:13.618567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 852/1709] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:11 2022] POST /identity/v3/users => generated 287 bytes in 1990 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:13.622142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:13.658740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db7824-480c-4f69-9a1e-b248f4013fec service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:13.675816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:13.675816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:13.675816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:13.675816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:13.676370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:13.685252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:13.707537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 858/1710] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:13.707877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:13.723267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22072-34a1-4ab8-b952-e97bf92e0899 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:13.729592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a31755-9270-479f-b068-1f9595b9feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:13.730691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31755-9270-479f-b068-1f9595b9feaf admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:13.731267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31755-9270-479f-b068-1f9595b9feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:13.731777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31755-9270-479f-b068-1f9595b9feaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:13.732719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a31755-9270-479f-b068-1f9595b9feaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:13.734654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a31755-9270-479f-b068-1f9595b9feaf admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:13.758955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 853/1711] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:13.769227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:13.784910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a4d1c-ea5e-4963-8fbf-e115ff93c46c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:13.791948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:13.792992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:13.794419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:13.795072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] PATH_INFO: `/v3/projects/47420028e9864638ab8c5f74f2180ae8/users/33d2726f5ead40b99c00cc76f86a20a9/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:13.799428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:13.866756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47420028e9864638ab8c5f74f2180ae8, user_id=33d2726f5ead40b99c00cc76f86a20a9, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:13.875571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:14.043228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 859/1712] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:13 2022] PUT /identity/v3/projects/47420028e9864638ab8c5f74f2180ae8/users/33d2726f5ead40b99c00cc76f86a20a9/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:14.055831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:14.056086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:14.056340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:14.186807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] MFA Rules not processed for user `33d2726f5ead40b99c00cc76f86a20a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:14.190433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:14.551984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 854/1713] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:14.557353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:14.811799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff50b2-5b06-4e10-ab5e-e11efcad5480 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:14.827417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20d6d-3ea4-4109-93d0-fbce704aa5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:14.831032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20d6d-3ea4-4109-93d0-fbce704aa5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:14.832250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20d6d-3ea4-4109-93d0-fbce704aa5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:14.833187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20d6d-3ea4-4109-93d0-fbce704aa5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:14.837225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c20d6d-3ea4-4109-93d0-fbce704aa5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:14.839302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c20d6d-3ea4-4109-93d0-fbce704aa5e1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:15.048516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 860/1714] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:14 2022] POST /identity/v3/projects => generated 371 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:15.051629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:15.114606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223ea1-12a3-4da3-89b2-3aee36c40ac9 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:15.153655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:15.184681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a5121-1f86-430f-ab30-66fcf299451f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:15.185025 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a5121-1f86-430f-ab30-66fcf299451f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:15.185317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a5121-1f86-430f-ab30-66fcf299451f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:15.277547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6e2a5121-1f86-430f-ab30-66fcf299451f None None] MFA Rules not processed for user `7562a0429fdf42948174b517b211884f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:15.278868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e2a5121-1f86-430f-ab30-66fcf299451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:15.285301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:15.286805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:15.287304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:15.288151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:15.289118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:15.292248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:15.418232 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 855/1715] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:15 2022] POST /identity/v3/users => generated 292 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:15.422228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:15.490177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e87ccc-af9d-42a3-b06c-a58c3fad411f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:15.513288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 861/1716] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:15.513288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb666b6e-0237-4f11-bd71-187bc1c1fd5c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:15.513288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb666b6e-0237-4f11-bd71-187bc1c1fd5c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:15.513288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb666b6e-0237-4f11-bd71-187bc1c1fd5c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:15.515247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 862/1717] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:06:15.524422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:15.524422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:15.524422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:15.526189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:15.527234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:15.540898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:15.568361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:15.568955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:15.569345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:15.638942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 856/1718] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:15.643603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:15.715251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:15.718886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b58368-ec0e-42f7-8ef6-f6b5afe9778d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:15.724765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:15.756275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:15.764670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:15.766536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:15.766536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] PATH_INFO: `/v3/projects/48a4cb7140d44a71853e90892071fb76/users/ed79af3dd2614fe7a0dc2c7133d77015/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:15.886812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a4cb7140d44a71853e90892071fb76, user_id=ed79af3dd2614fe7a0dc2c7133d77015, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:15.913360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:15.978389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 863/1719] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:16.172562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 857/1720] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:15 2022] PUT /identity/v3/projects/48a4cb7140d44a71853e90892071fb76/users/ed79af3dd2614fe7a0dc2c7133d77015/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:16.174777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:16.183077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:16.195897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1236e-da68-4622-afc0-5b2fe476ba89 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:16.382807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:16.385620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:16.401615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:16.404522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:16.405177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:16.406114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:16.409882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:16.416017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:16.429083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff6af9-4c4f-4d5f-8908-bd4ce69a7051 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:16.459977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:16.461876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:16.464455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:16.466204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] PATH_INFO: `/v3/users/8b1b6818b654453e90c2cdfd5cb04d4a` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:16.467484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 864/1721] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:16.472465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:16.473823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1b6818b654453e90c2cdfd5cb04d4a)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:16.483324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:16.515341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61fd0-29ea-42a8-8106-aac922e98dd1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:16.532846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6663b8b-1ace-4d16-ba19-8b3cd0629c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:16.533399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6663b8b-1ace-4d16-ba19-8b3cd0629c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:16.534015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6663b8b-1ace-4d16-ba19-8b3cd0629c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:16.537724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6663b8b-1ace-4d16-ba19-8b3cd0629c29 admin admin] PATH_INFO: `/v3/projects/48a4cb7140d44a71853e90892071fb76/users/ed79af3dd2614fe7a0dc2c7133d77015/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:16.539691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6663b8b-1ace-4d16-ba19-8b3cd0629c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a4cb7140d44a71853e90892071fb76, user_id=ed79af3dd2614fe7a0dc2c7133d77015, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:16.551178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6663b8b-1ace-4d16-ba19-8b3cd0629c29 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:16.596867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 865/1722] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:16 2022] PUT /identity/v3/projects/48a4cb7140d44a71853e90892071fb76/users/ed79af3dd2614fe7a0dc2c7133d77015/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:16.598828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:16.599124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:16.599411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:16.668513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] MFA Rules not processed for user `ed79af3dd2614fe7a0dc2c7133d77015`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:16.669935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:16.916840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 866/1723] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:16.924650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:16.936502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:16.982802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1b6818b654453e90c2cdfd5cb04d4a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:17.106314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1b6818b654453e90c2cdfd5cb04d4a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:17.130772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3483f-fcf6-4d30-b427-e71f928a768c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:17.132562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1b6818b654453e90c2cdfd5cb04d4a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:17.133471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1b6818b654453e90c2cdfd5cb04d4a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:17.145257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c5cc2272-309d-4185-b46b-db823b299198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1b6818b654453e90c2cdfd5cb04d4a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:17.160359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:17.161826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 858/1724] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:16 2022] DELETE /identity/v3/users/8b1b6818b654453e90c2cdfd5cb04d4a => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:17.163598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:17.164188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:17.164827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:17.168567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee5fc5-e147-480d-b473-1994ea7d34c6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:17.169465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee5fc5-e147-480d-b473-1994ea7d34c6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:17.169967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee5fc5-e147-480d-b473-1994ea7d34c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:17.171669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:17.208328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-48ee5fc5-e147-480d-b473-1994ea7d34c6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:17.209865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-48ee5fc5-e147-480d-b473-1994ea7d34c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:17.216604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:17.234714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:17.301295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 867/1725] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:17.380434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:17.408575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 859/1726] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:17.424127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:17.426534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:17.655372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6832522-c5b8-470f-a365-c229ec2ee40d service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:17.719530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:17.764979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:17.766464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:17.767074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:17.768340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:17.787274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:17.790868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:17.810178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 868/1727] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:17.896436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:17.896923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:17.897360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:17.941584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] MFA Rules not processed for user `ff664c7ba095413daad6b3f1ebc49cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:17.944270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:18.319995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 860/1728] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:18.371538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:18.373316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:18.665830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43bab-3096-4b0b-83b9-06f014a599ad service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:18.701257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:18.702969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:18.709210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:18.710172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:18.714896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:18.778037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:18.801402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:18.853965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 869/1729] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:18.909599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:18.912698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:18.968138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf6808-a061-4397-a5cd-a9bfcaa1d889 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:18.996157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:18.998257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:18.998966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:19.003054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] PATH_INFO: `/v3/projects/16b26a6f01844176b377642e6028d667` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:19.028127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b26a6f01844176b377642e6028d667)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:19.050617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:19.406479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b26a6f01844176b377642e6028d667', 'request_id': 'req-903e62f2-68c7-4b77-944c-938562a9c931'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:19.425455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b26a6f01844176b377642e6028d667', 'request_id': 'req-903e62f2-68c7-4b77-944c-938562a9c931'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:19.426255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:19.428175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:19.476195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014834-5b56-4770-ab5f-f99f0a70ed7d service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:19.571889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b26a6f01844176b377642e6028d667', 'request_id': 'req-903e62f2-68c7-4b77-944c-938562a9c931'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:19.648799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] The token cache is being invalidate because project 16b26a6f01844176b377642e6028d667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:19.660683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:19.660683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:19.669627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 861/1730] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:18 2022] DELETE /identity/v3/projects/16b26a6f01844176b377642e6028d667 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:19.671349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:19.679105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:19.885984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:19.887047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:19.887500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:19.888774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] PATH_INFO: `/v3/users/33d2726f5ead40b99c00cc76f86a20a9` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:19.895650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d2726f5ead40b99c00cc76f86a20a9)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:19.906494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:19.913209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:20.068353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e62f2-68c7-4b77-944c-938562a9c931 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:20.077769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:20.083607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:20.083607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:20.083607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] PATH_INFO: `/v3/users/b797a0d684494aa691ba73dd4388bd69` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:20.089821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b797a0d684494aa691ba73dd4388bd69)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:20.094869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:20.111210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:20.324835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d2726f5ead40b99c00cc76f86a20a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.326167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d2726f5ead40b99c00cc76f86a20a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.371355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d2726f5ead40b99c00cc76f86a20a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.489870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d2726f5ead40b99c00cc76f86a20a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.541560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-28da8705-8bbf-4a01-955a-0e4fdd5c9e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d2726f5ead40b99c00cc76f86a20a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.554386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b797a0d684494aa691ba73dd4388bd69'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.637727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 870/1731] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:19 2022] DELETE /identity/v3/users/33d2726f5ead40b99c00cc76f86a20a9 => generated 0 bytes in 1212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:20.651468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:20.662105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:20.663423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:20.740986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b797a0d684494aa691ba73dd4388bd69'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.775357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:20.781187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:20.791335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b797a0d684494aa691ba73dd4388bd69'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.792696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b797a0d684494aa691ba73dd4388bd69'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.821550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b797a0d684494aa691ba73dd4388bd69'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:20.862597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 862/1732] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:19 2022] DELETE /identity/v3/users/b797a0d684494aa691ba73dd4388bd69 => generated 0 bytes in 1192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:21.029889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 871/1733] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:21.044811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:21.048085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:21.143598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:21.146072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:21.364050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e706d-10ef-48fb-b560-67261ca8cb62 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:21.374114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:21.374909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:21.377620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:21.380002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] PATH_INFO: `/v3/users/3d3a5ecdf0a24e079a307b5124b6131b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:21.382902 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3a5ecdf0a24e079a307b5124b6131b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:21.384807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1ce3b-df1c-4b3c-88ac-0c549456c386 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:21.398234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:21.557073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:21.640997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f39f6-ef94-4cb3-abd7-1948ea047756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:21.642450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f39f6-ef94-4cb3-abd7-1948ea047756 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:21.645119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f39f6-ef94-4cb3-abd7-1948ea047756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:21.647090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f39f6-ef94-4cb3-abd7-1948ea047756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:21.679712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f39f6-ef94-4cb3-abd7-1948ea047756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:21.683384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f39f6-ef94-4cb3-abd7-1948ea047756 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:21.732140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 872/1734] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:21.748849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3a5ecdf0a24e079a307b5124b6131b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:21.837190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3a5ecdf0a24e079a307b5124b6131b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:21.890348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3a5ecdf0a24e079a307b5124b6131b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:21.894152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3a5ecdf0a24e079a307b5124b6131b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:21.930102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3a5ecdf0a24e079a307b5124b6131b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:21.974119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 863/1735] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:21 2022] DELETE /identity/v3/users/3d3a5ecdf0a24e079a307b5124b6131b => generated 0 bytes in 929 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:21.976597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:21.977120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:21.977756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:22.055055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:22.056778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:22.224405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 873/1736] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:22.506537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:22.514044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:22.541118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:22.707822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3c38a-6d90-455a-b1cc-83d569ff9cd1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:22.727876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:22.730094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:22.731295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:22.732338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:22.794152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:22.802405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:22.846002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 864/1737] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:23.614890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:23.616756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:23.634675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4584396-0689-47d2-b3b9-7b9e32e5dbab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:23.660542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:23.662464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:23.678964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:23.683518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:23.684287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:23.684649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] PATH_INFO: `/v3/projects/60a7fdd5497143a9b2484956115dc1fa` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:23.689607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a7fdd5497143a9b2484956115dc1fa)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:23.696949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:23.702900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c7467-8d99-4aea-bcf3-d83d0b05c09f service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:23.725716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:23.726442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:23.726719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:23.727984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] PATH_INFO: `/v3/projects/47420028e9864638ab8c5f74f2180ae8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:23.731744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47420028e9864638ab8c5f74f2180ae8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:23.735485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:23.951468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47420028e9864638ab8c5f74f2180ae8', 'request_id': 'req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:23.970864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47420028e9864638ab8c5f74f2180ae8', 'request_id': 'req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:23.981260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a7fdd5497143a9b2484956115dc1fa', 'request_id': 'req-f4abf432-7bf3-49dc-a706-ff00b1c456c9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.022115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a7fdd5497143a9b2484956115dc1fa', 'request_id': 'req-f4abf432-7bf3-49dc-a706-ff00b1c456c9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.044456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a7fdd5497143a9b2484956115dc1fa', 'request_id': 'req-f4abf432-7bf3-49dc-a706-ff00b1c456c9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.117242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47420028e9864638ab8c5f74f2180ae8', 'request_id': 'req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.137057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] The token cache is being invalidate because project 47420028e9864638ab8c5f74f2180ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:24.137345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.140009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.147076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 865/1738] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:23 2022] DELETE /identity/v3/projects/47420028e9864638ab8c5f74f2180ae8 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:24.152183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:24.154105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:24.357011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af8b2d-ac1e-4a9e-b85a-9b062c62c41a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:24.373455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:24.375968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:24.377487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:24.377738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] PATH_INFO: `/v3/users/ed79af3dd2614fe7a0dc2c7133d77015` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:24.379567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed79af3dd2614fe7a0dc2c7133d77015)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:24.381567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:24.418830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] The token cache is being invalidate because project 60a7fdd5497143a9b2484956115dc1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:24.424062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.427597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f4abf432-7bf3-49dc-a706-ff00b1c456c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.437384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 874/1739] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:23 2022] DELETE /identity/v3/projects/60a7fdd5497143a9b2484956115dc1fa => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:24.446809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed789f2-7aac-4cad-bbb6-317bdaa3ae12 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:24.448314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed789f2-7aac-4cad-bbb6-317bdaa3ae12 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:24.448758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed789f2-7aac-4cad-bbb6-317bdaa3ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:24.582263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4ed789f2-7aac-4cad-bbb6-317bdaa3ae12 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:24.584589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4ed789f2-7aac-4cad-bbb6-317bdaa3ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:24.630462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 875/1740] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:24.639402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7822e-d164-4052-8375-39a985180d18 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:24.640061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7822e-d164-4052-8375-39a985180d18 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:24.640585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7822e-d164-4052-8375-39a985180d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:24.679989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-47b7822e-d164-4052-8375-39a985180d18 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:24.681472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-47b7822e-d164-4052-8375-39a985180d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:24.705728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 876/1741] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:24.710662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7822e-d164-4052-8375-39a985180d18 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:24.716596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:24.724293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7822e-d164-4052-8375-39a985180d18 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:24.731369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:24.732571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:24.733175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:24.733612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:24.734682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:24.735185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:24.737763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:24.763109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 877/1742] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:24.765423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:24.777043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed79af3dd2614fe7a0dc2c7133d77015'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.827036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a59c7-c9bc-4c6f-89b7-f5f82376c267 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:24.848776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:24.849792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:24.850302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:24.850756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:24.854402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:24.856768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:24.883869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed79af3dd2614fe7a0dc2c7133d77015'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.901515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed79af3dd2614fe7a0dc2c7133d77015'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.902308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed79af3dd2614fe7a0dc2c7133d77015'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.911584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed79af3dd2614fe7a0dc2c7133d77015'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:24.919261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 866/1743] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:24 2022] DELETE /identity/v3/users/ed79af3dd2614fe7a0dc2c7133d77015 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:25.019472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.021638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:25.039631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 878/1744] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:24 2022] POST /identity/v3/projects => generated 367 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:25.042294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.047106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:25.084352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:25.168310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a195-8b41-470a-9bd4-3019fe98ff67 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.174716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.175530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.175822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.176218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.177101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.179354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:25.211337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8c0bd-7c74-4581-b1a4-e5d5beea8eec admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.217818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.218826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.219293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.226335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] PATH_INFO: `/v3/projects/1420fdd935654e00b1da064f02b5f1ae` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.226335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1420fdd935654e00b1da064f02b5f1ae)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.226335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:25.289945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 879/1745] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:25 2022] POST /identity/v3/users => generated 284 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:25.291121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.318954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b676e0-00d1-4122-9451-2ec22843857a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.332898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.333973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.334269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.334541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.335271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.342310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:25.358992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 880/1746] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:25.389186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.418275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06edf7-f1f0-4fc9-8bb8-616976923d2b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.419286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1420fdd935654e00b1da064f02b5f1ae', 'request_id': 'req-a4ab09fa-cd5e-41be-b91e-6b777659cd21'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:25.440482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1420fdd935654e00b1da064f02b5f1ae', 'request_id': 'req-a4ab09fa-cd5e-41be-b91e-6b777659cd21'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:25.534142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1420fdd935654e00b1da064f02b5f1ae', 'request_id': 'req-a4ab09fa-cd5e-41be-b91e-6b777659cd21'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:25.541741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.543801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.544088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.544454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.545167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.546881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:25.559665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] The token cache is being invalidate because project 1420fdd935654e00b1da064f02b5f1ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:25.560209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:25.561368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:25.567699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 867/1747] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:25 2022] DELETE /identity/v3/projects/1420fdd935654e00b1da064f02b5f1ae => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:25.569443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.569989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 881/1748] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:25.573707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.574895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:25.576480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:25.613583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a982e8-dde0-4f51-8c2c-87aa10578600 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.623430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.636036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab09fa-cd5e-41be-b91e-6b777659cd21 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.636249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.636249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.636249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] PATH_INFO: `/v3/projects/89570a08d3ea428bad8858c7128c3f73/users/720baba122af425880e1c22ab482aa69/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.648825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.649869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.650481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.650981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] PATH_INFO: `/v3/users/abf8dfa5e1a2481fb6cb3c5a2e47b7d0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.653229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf8dfa5e1a2481fb6cb3c5a2e47b7d0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.657408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:25.674905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89570a08d3ea428bad8858c7128c3f73, user_id=720baba122af425880e1c22ab482aa69, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.677821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:25.788129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 882/1749] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:25 2022] PUT /identity/v3/projects/89570a08d3ea428bad8858c7128c3f73/users/720baba122af425880e1c22ab482aa69/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:25.788129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:25.797448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:25.970701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf8dfa5e1a2481fb6cb3c5a2e47b7d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:25.975176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b381542-9861-4e31-87d9-f0af8abf3a5c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:25.983349 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:25.984502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:25.989433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:25.989884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] PATH_INFO: `/v3/projects/48a4cb7140d44a71853e90892071fb76` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:25.991633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a4cb7140d44a71853e90892071fb76)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:25.994569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:26.065973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf8dfa5e1a2481fb6cb3c5a2e47b7d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.082470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf8dfa5e1a2481fb6cb3c5a2e47b7d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.083604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf8dfa5e1a2481fb6cb3c5a2e47b7d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.092416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-1b20d622-3e08-411d-a3af-f02795c2889f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf8dfa5e1a2481fb6cb3c5a2e47b7d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.104292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 868/1750] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:25 2022] DELETE /identity/v3/users/abf8dfa5e1a2481fb6cb3c5a2e47b7d0 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:26.104292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.104292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.105885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.165476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] MFA Rules not processed for user `720baba122af425880e1c22ab482aa69`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:26.167495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:26.187208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a4cb7140d44a71853e90892071fb76', 'request_id': 'req-a1904d02-72f5-4ca5-b685-37d979071d93'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.198287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a4cb7140d44a71853e90892071fb76', 'request_id': 'req-a1904d02-72f5-4ca5-b685-37d979071d93'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.208978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a4cb7140d44a71853e90892071fb76', 'request_id': 'req-a1904d02-72f5-4ca5-b685-37d979071d93'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.271743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 869/1751] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:26.276974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:26.278476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:26.325180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] The token cache is being invalidate because project 48a4cb7140d44a71853e90892071fb76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:26.325878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.327928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a1904d02-72f5-4ca5-b685-37d979071d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:26.332124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 883/1752] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:25 2022] DELETE /identity/v3/projects/48a4cb7140d44a71853e90892071fb76 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:26.338857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d733-7a1e-432b-8049-e23bd2d0969e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.339474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d733-7a1e-432b-8049-e23bd2d0969e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.339980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d733-7a1e-432b-8049-e23bd2d0969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.386838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:26.389437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7cf1d733-7a1e-432b-8049-e23bd2d0969e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:26.391609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7cf1d733-7a1e-432b-8049-e23bd2d0969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:26.435112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:26.461102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97773d47-de19-4256-b277-1fa93608fc8e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:26.479889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:26.480581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.480885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.481164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.481882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:26.484578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:26.587577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 884/1753] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:26.600114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.600874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.601342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.641055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 870/1754] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:26 2022] POST /identity/v3/projects => generated 367 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:26.649071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:26.659921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:26.663403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:26.670754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:26.761756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 885/1755] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:26.788457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:26.803558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20b959-4f11-403f-bfe7-7304f17f96de admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:26.806852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1c4e-3f10-4a99-a32d-3a3670020c51 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:26.813705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:26.815675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.816428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.817112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.819241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:26.820680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:26.821506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.821970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.822409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.823229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:26.823680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:26.824057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:26.825868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:26.841419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 886/1756] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:26.848497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:26.880838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19447c7-e388-4f16-a142-f79d5f19a7ac admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:26.887072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:26.887072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:26.887072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:26.887072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:26.887760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:26.888452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:26.977483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 871/1757] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:26 2022] POST /identity/v3/users => generated 290 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:26.981528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:26.993381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f7e8-eaab-4130-9e1a-2eb63f8f4bb6 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:26.999196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:27.000133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.001041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.001928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.002659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:27.004312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:27.011721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:27.018036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 887/1758] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:26 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:27.024175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:27.035157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 872/1759] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:27.036409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:27.037672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:27.050148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55522b-b7da-4602-bddd-5315cb83f61c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:27.199823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd39f17-0bd9-4d2d-b40b-6f8cbaf771bf admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:27.213597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:27.214485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.214770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.215118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.215763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:27.218009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:27.281971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:27.282793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.285796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.286759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] PATH_INFO: `/v3/projects/bad242d64e9a44caa7164886f992ff32/users/0e3df0a5ca62414889722bad60e7e624/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.326064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad242d64e9a44caa7164886f992ff32, user_id=0e3df0a5ca62414889722bad60e7e624, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:27.331279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:27.434044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 888/1760] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:27 2022] POST /identity/v3/users => generated 281 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:27.435179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:27.438203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:27.461372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9941-272d-4887-b2cc-0a6c3b98ea43 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:27.471099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:27.482493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.483534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.485240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] PATH_INFO: `/v3/projects/2dbde8c1a1da4245a21f7b70af70af82` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.500381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dbde8c1a1da4245a21f7b70af70af82)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:27.505847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:27.535818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 873/1761] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:27 2022] PUT /identity/v3/projects/bad242d64e9a44caa7164886f992ff32/users/0e3df0a5ca62414889722bad60e7e624/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:27.542164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:27.665021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bce92-7211-4a5a-bdf0-5676f79b87cc admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:27.670202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:27.670849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.671134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.671514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.672183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:27.674736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:27.698898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 874/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:27.699945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:27.715700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ed59-2c18-4957-86c4-6ad02f3b075c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:27.730244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dbde8c1a1da4245a21f7b70af70af82', 'request_id': 'req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:27.764633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dbde8c1a1da4245a21f7b70af70af82', 'request_id': 'req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:27.780152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dbde8c1a1da4245a21f7b70af70af82', 'request_id': 'req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:27.855334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] The token cache is being invalidate because project 2dbde8c1a1da4245a21f7b70af70af82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:27.855728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:27.856407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:27.860147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 889/1763] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:27 2022] DELETE /identity/v3/projects/2dbde8c1a1da4245a21f7b70af70af82 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:27.860147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:27.861689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:27.900929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e262da-d0c1-4820-b2e0-408cb4b05267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:27.902257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e262da-d0c1-4820-b2e0-408cb4b05267 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.902861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e262da-d0c1-4820-b2e0-408cb4b05267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.903452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e262da-d0c1-4820-b2e0-408cb4b05267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.905356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e262da-d0c1-4820-b2e0-408cb4b05267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:27.910716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e262da-d0c1-4820-b2e0-408cb4b05267 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:27.933839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 875/1764] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:27.937192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e73988-cd4c-451a-af6b-44588bf6b7a2 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.937871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e73988-cd4c-451a-af6b-44588bf6b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.938697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e73988-cd4c-451a-af6b-44588bf6b7a2 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.942900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 876/1765] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:06:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:06:27.946108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:27.946701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:27.947303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:27.993855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca613f-a6db-4b0f-9a73-ea9f59cbd00b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:27.996869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:27.998445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.008616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ee2f5-ab92-4eca-a470-18b439702452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.009978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ee2f5-ab92-4eca-a470-18b439702452 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.010716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ee2f5-ab92-4eca-a470-18b439702452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.011585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ee2f5-ab92-4eca-a470-18b439702452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.015492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 877/1766] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:28.015885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ee2f5-ab92-4eca-a470-18b439702452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.022346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:28.024398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.024647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ee2f5-ab92-4eca-a470-18b439702452 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.065466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c8a60-9fae-432c-83be-85f0d4e1b231 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:28.091151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1b90e-6768-484c-a8d6-4b440426cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.092018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b90e-6768-484c-a8d6-4b440426cfc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.092475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b90e-6768-484c-a8d6-4b440426cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.093664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b90e-6768-484c-a8d6-4b440426cfc3 admin admin] PATH_INFO: `/v3/projects/bad242d64e9a44caa7164886f992ff32/users/0e3df0a5ca62414889722bad60e7e624/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.101972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 890/1767] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:28.102445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.102943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.103424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.107370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1b90e-6768-484c-a8d6-4b440426cfc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad242d64e9a44caa7164886f992ff32, user_id=0e3df0a5ca62414889722bad60e7e624, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.110658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1b90e-6768-484c-a8d6-4b440426cfc3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.134729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:28.172258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 878/1768] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:28 2022] PUT /identity/v3/projects/bad242d64e9a44caa7164886f992ff32/users/0e3df0a5ca62414889722bad60e7e624/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:28.172258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164336-d90f-4f2f-b5f5-3f409021b2e1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.172258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164336-d90f-4f2f-b5f5-3f409021b2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.172258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164336-d90f-4f2f-b5f5-3f409021b2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.180689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:28.180689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.192514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:28.221626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-58164336-d90f-4f2f-b5f5-3f409021b2e1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:28.224628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-58164336-d90f-4f2f-b5f5-3f409021b2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.393855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 891/1769] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:06:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:28.398529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:28.419522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 879/1770] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:28.428324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.429314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.429314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.464348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] MFA Rules not processed for user `0e3df0a5ca62414889722bad60e7e624`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:28.466266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.505934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a765ba-f3fc-4f8f-824f-8f13bce269fc None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:28.510928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.511931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.512195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.512471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] PATH_INFO: `/v3/projects/02d0ef28230e4e938d59bc2b389f0c1b/users/2ca0bd2bc636418daf59075a644f0bfd/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.529131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d0ef28230e4e938d59bc2b389f0c1b, user_id=2ca0bd2bc636418daf59075a644f0bfd, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.532004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.564645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 880/1771] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:28.565660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:28.567533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.592574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9999c34-5fe3-42c3-ba6d-220bd1517e22 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:28.600968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.602161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.602761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.604553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.615583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.617228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.630071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 881/1772] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:28.630693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:28.643302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c76c2-394a-4ee2-82bf-f12797828fed service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:28.648974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.649607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.649885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.650266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.651113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:28.651395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.653312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.669779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 882/1773] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:28.670675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:28.679497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:28.707399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 892/1774] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:28 2022] PUT /identity/v3/projects/02d0ef28230e4e938d59bc2b389f0c1b/users/2ca0bd2bc636418daf59075a644f0bfd/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:28.707399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:28.800011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eb627-6f47-41ff-a6ae-92ea80d6351a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:28.881217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3beec-4fc0-447e-ae7d-2d0abbab4c71 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:28.887095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.888074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.888074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.888285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.890957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.891083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f933df-e01e-4f8d-ab51-f2d5daf7b072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:28.891196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.892424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f933df-e01e-4f8d-ab51-f2d5daf7b072 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.892957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f933df-e01e-4f8d-ab51-f2d5daf7b072 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.893481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f933df-e01e-4f8d-ab51-f2d5daf7b072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:28.971102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f933df-e01e-4f8d-ab51-f2d5daf7b072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:28.972696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f933df-e01e-4f8d-ab51-f2d5daf7b072 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:28.984083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 883/1775] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:28.985526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:28.986385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:28.994102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.024644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] MFA Rules not processed for user `2ca0bd2bc636418daf59075a644f0bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:29.035650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:29.046360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 893/1776] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:28 2022] POST /identity/v3/projects => generated 385 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:29.051370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.136324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:29.208709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 884/1777] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:29.208709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.213353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc86240-afcd-4a4d-ba16-c299f9139f9f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.223853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.224736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.225080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.225443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.226453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.228767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.236840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a642e6e-58c0-4d3f-8510-a05d0771fd19 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.334485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.335547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.336060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.336652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.339858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:29.372085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 894/1778] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:29 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:29.373835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.414262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec5db-1a11-4a81-9d7e-0fd7f157cfa5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.430559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.432054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.432571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.433073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.435192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.444273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.480795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.482969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.502143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 885/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:29.513002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.529433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54632614-0176-41bf-a6f0-3e46a3a6ca7a service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.535725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.536698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.537060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.537449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.543032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.547042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.581211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 886/1780] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:29.583448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.606083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238aee-0a42-4163-889a-d3204dac5322 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.606778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 895/1781] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:29 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:29.609773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.679077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.679793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.680396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.680396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.686894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.686894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.698949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 887/1782] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:29.704076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.716288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357e6e9-9286-4304-ae4d-3b9712cfa06e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.723489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb2866-15a6-40bc-b1b9-cb2ef2505894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.724441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2866-15a6-40bc-b1b9-cb2ef2505894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.724792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2866-15a6-40bc-b1b9-cb2ef2505894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.725128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2866-15a6-40bc-b1b9-cb2ef2505894 admin admin] PATH_INFO: `/v3/projects/8cdeb25f5b23482a81a5326789a5d4e0/users/8128b399d4dd4e17be04c537d66d1b2b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.731780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1aa66-ae5e-4fd4-a761-e199888688e1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.738583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.739322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.739676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.740030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.740950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.743067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.749584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb2866-15a6-40bc-b1b9-cb2ef2505894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdeb25f5b23482a81a5326789a5d4e0, user_id=8128b399d4dd4e17be04c537d66d1b2b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.753118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb2866-15a6-40bc-b1b9-cb2ef2505894 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:29.897775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 896/1783] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:29 2022] POST /identity/v3/users => generated 289 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:29.903411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:29.910479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 888/1784] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:29 2022] PUT /identity/v3/projects/8cdeb25f5b23482a81a5326789a5d4e0/users/8128b399d4dd4e17be04c537d66d1b2b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:29.912729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.913233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.913681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.918040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f65d-aac2-4291-80dc-e9e5a0c23f58 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:29.936838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:29.956280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] MFA Rules not processed for user `8128b399d4dd4e17be04c537d66d1b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:29.957553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:29.993808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:29.994697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:29.995078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:29.995605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:29.996522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:29.999068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.011974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 897/1785] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:30.017292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.029999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dea3f-966c-4147-a7ed-e74e2068d4f9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.036538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:30.037563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.037870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.038166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] PATH_INFO: `/v3/projects/056fdf4d6eae49809b29e4a9268abe90/users/aa2ed3261fbd4d3f8f034f2edca1dbe7/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.043479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 889/1786] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:30.049514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.061883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480ab23-46a6-47ea-88db-799d726db864 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.068182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:30.068813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.069097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.069393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=056fdf4d6eae49809b29e4a9268abe90, user_id=aa2ed3261fbd4d3f8f034f2edca1dbe7, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:30.069565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.070076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:30.072106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.073393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.231303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 898/1787] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:30 2022] PUT /identity/v3/projects/056fdf4d6eae49809b29e4a9268abe90/users/aa2ed3261fbd4d3f8f034f2edca1dbe7/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:30.237735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.250780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 890/1788] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:30 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:30.262549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.418006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54429c-528d-4b76-96fa-896fc90f5d7c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.499456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0a26c-5724-42bb-9cbe-aea97b24c059 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.531379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:30.532029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.532314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.532634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.548606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:30.565402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.652795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:30.653854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.654822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.654961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.661000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:30.661166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:30.662473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.701962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 899/1789] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:30.708922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.729287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 891/1790] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:30 2022] POST /identity/v3/users => generated 300 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:30.734169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451547c6-7e98-4a10-bd7b-997362eecbdd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.736794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.749329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b862-cc38-450d-a08c-b32f26ca212b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.752690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f372164-af8c-4427-a9b7-11197227ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:30.754254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372164-af8c-4427-a9b7-11197227ff68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.755033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372164-af8c-4427-a9b7-11197227ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.755774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372164-af8c-4427-a9b7-11197227ff68 admin admin] PATH_INFO: `/v3/projects/056fdf4d6eae49809b29e4a9268abe90/users/aa2ed3261fbd4d3f8f034f2edca1dbe7/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.759022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:30.759764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.760086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.760492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.761467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:30.762745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372164-af8c-4427-a9b7-11197227ff68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056fdf4d6eae49809b29e4a9268abe90, user_id=aa2ed3261fbd4d3f8f034f2edca1dbe7, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:30.778369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372164-af8c-4427-a9b7-11197227ff68 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.791666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:30.803155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 900/1791] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:30 2022] PUT /identity/v3/projects/056fdf4d6eae49809b29e4a9268abe90/users/aa2ed3261fbd4d3f8f034f2edca1dbe7/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:30.817762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:30.818577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:30.819072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:30.822015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 892/1792] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:30.831018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:30.854771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185f07-b144-4ce8-8ef1-c261c922b286 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:30.872875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] MFA Rules not processed for user `aa2ed3261fbd4d3f8f034f2edca1dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:30.874905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:31.005504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 901/1793] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:31.039215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:31.040219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:31.042898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:31.049049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] PATH_INFO: `/v3/projects/96c035e94b9141b98fddc7aba982d815/users/3d0a14e764924da48d3ae13b2c9e0110/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:31.049861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:31.108836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c035e94b9141b98fddc7aba982d815, user_id=3d0a14e764924da48d3ae13b2c9e0110, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:31.123983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:31.231654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 893/1794] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:30 2022] PUT /identity/v3/projects/96c035e94b9141b98fddc7aba982d815/users/3d0a14e764924da48d3ae13b2c9e0110/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:31.233334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:31.244284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94962-9de8-4e8f-895d-64cfb6fe43d8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:31.311635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:31.438772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:31.471392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ca725-de22-4528-9ce7-9ecbde65631f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:31.472513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ca725-de22-4528-9ce7-9ecbde65631f service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:31.472979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ca725-de22-4528-9ce7-9ecbde65631f service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:31.473433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ca725-de22-4528-9ce7-9ecbde65631f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:31.549685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13555cdd-4f71-4769-a70e-1237b5ed862f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:31.570222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:31.571313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:31.571820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:31.572856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:31.617368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:31.620685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:31.645526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 894/1795] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:31.652056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ca725-de22-4528-9ce7-9ecbde65631f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:31.652996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:31.656531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ca725-de22-4528-9ce7-9ecbde65631f service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:31.670317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee769c-573a-43d0-b4b8-9bba6188a360 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:31.677246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:31.678140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:31.678821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:31.679343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] PATH_INFO: `/v3/projects/96c035e94b9141b98fddc7aba982d815/users/3d0a14e764924da48d3ae13b2c9e0110/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:31.682046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c035e94b9141b98fddc7aba982d815, user_id=3d0a14e764924da48d3ae13b2c9e0110, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:31.683380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 902/1796] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:31.685131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:31.701144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 895/1797] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:31 2022] PUT /identity/v3/projects/96c035e94b9141b98fddc7aba982d815/users/3d0a14e764924da48d3ae13b2c9e0110/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:31.721138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:31.721522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:31.721852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:31.758814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] MFA Rules not processed for user `3d0a14e764924da48d3ae13b2c9e0110`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:31.760088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:31.893533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 903/1798] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:31.911830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:32.004107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d582a-173b-4980-b85a-a1f138b2ab07 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:32.011591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:32.012662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:32.013175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:32.013578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:32.029014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:32.030929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:32.049196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 896/1799] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:32.539254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:32.560114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:32.564002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:32.647264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90ce8a-175a-454d-84ac-b0270e2cac12 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:32.666199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6054c0-800f-4c21-aed9-84607d429a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:32.673047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6054c0-800f-4c21-aed9-84607d429a1f service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:32.674824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6054c0-800f-4c21-aed9-84607d429a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:32.677475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6054c0-800f-4c21-aed9-84607d429a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:32.730674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b8e36-ea52-4490-b04e-7fd9f595834e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:32.740467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:32.742744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:32.743695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:32.744792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] PATH_INFO: `/v3/users/8128b399d4dd4e17be04c537d66d1b2b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:32.748548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8128b399d4dd4e17be04c537d66d1b2b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:32.753205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:32.839992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6054c0-800f-4c21-aed9-84607d429a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:32.847942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6054c0-800f-4c21-aed9-84607d429a1f service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:32.883945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 897/1800] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:33.126180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8128b399d4dd4e17be04c537d66d1b2b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:33.128584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8128b399d4dd4e17be04c537d66d1b2b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:33.155556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8128b399d4dd4e17be04c537d66d1b2b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:33.296095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8128b399d4dd4e17be04c537d66d1b2b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:33.317822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8128b399d4dd4e17be04c537d66d1b2b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:33.335377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 904/1801] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:32 2022] DELETE /identity/v3/users/8128b399d4dd4e17be04c537d66d1b2b => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:33.344675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:33.344675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:33.344675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:33.415404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:33.422818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:33.623207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 898/1802] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:33.690511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:33.694628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:33.904368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fe1f0-11ed-4dbc-a349-51cf250ba4ca admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:33.910809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:33.912539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:33.916996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:33.917589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:33.936799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:33.940843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:33.961585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 905/1803] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:34.369677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:34.537703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:35.268434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:35.270710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:35.285007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba42a0-3999-4c8c-bd8c-e36d787253da None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:35.294503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:35.299147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:35.299147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:35.299147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] PATH_INFO: `/v3/projects/8cdeb25f5b23482a81a5326789a5d4e0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:35.299147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cdeb25f5b23482a81a5326789a5d4e0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:35.309492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:35.538389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cdeb25f5b23482a81a5326789a5d4e0', 'request_id': 'req-527bbee2-01f8-4e79-926d-39936b4d1260'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:35.553998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cdeb25f5b23482a81a5326789a5d4e0', 'request_id': 'req-527bbee2-01f8-4e79-926d-39936b4d1260'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:35.685062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cdeb25f5b23482a81a5326789a5d4e0', 'request_id': 'req-527bbee2-01f8-4e79-926d-39936b4d1260'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:35.708298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] The token cache is being invalidate because project 8cdeb25f5b23482a81a5326789a5d4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:35.708638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:35.711277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-527bbee2-01f8-4e79-926d-39936b4d1260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:35.717534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:35.718127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 899/1804] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:35 2022] DELETE /identity/v3/projects/8cdeb25f5b23482a81a5326789a5d4e0 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:35.729664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845072a-0cf9-4486-ac38-e50970ef42f0 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:35.873105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:35.874718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:35.875733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:35.876975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] PATH_INFO: `/v3/users/3d0a14e764924da48d3ae13b2c9e0110` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:35.879835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0a14e764924da48d3ae13b2c9e0110)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:35.884702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:35.971103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427a07-707f-4245-857f-0a9726617086 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:35.971667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427a07-707f-4245-857f-0a9726617086 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:35.971959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427a07-707f-4245-857f-0a9726617086 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:35.973796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 900/1805] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:06:35.982548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:35.983078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:35.983509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:36.009283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:36.011100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:36.226652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0a14e764924da48d3ae13b2c9e0110'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:36.228627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0a14e764924da48d3ae13b2c9e0110'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:36.255866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 901/1806] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:06:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:36.274484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0a14e764924da48d3ae13b2c9e0110'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:36.306617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:36.309880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:36.360458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe8bfe-23bf-446a-99b2-1c17241c5426 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:36.379773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:36.384804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:36.384804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:36.384804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:36.401630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:36.407189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:36.453122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 902/1807] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:36.497887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0a14e764924da48d3ae13b2c9e0110'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:36.536145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8fa1e98f-d361-425b-af95-8b2a4fe06712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0a14e764924da48d3ae13b2c9e0110'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:36.575787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 906/1808] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:35 2022] DELETE /identity/v3/users/3d0a14e764924da48d3ae13b2c9e0110 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:38.187713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:38.190516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:38.217518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5306-8e51-4d4b-8800-61e0bdbf01d4 service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:38.348835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:38.350353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:38.350995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:38.351599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] PATH_INFO: `/v3/projects/96c035e94b9141b98fddc7aba982d815` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:38.354157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c035e94b9141b98fddc7aba982d815)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:38.389478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:38.507467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:38.638133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c035e94b9141b98fddc7aba982d815', 'request_id': 'req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:38.646442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c035e94b9141b98fddc7aba982d815', 'request_id': 'req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:38.654993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:38.763964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c035e94b9141b98fddc7aba982d815', 'request_id': 'req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:38.794973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] The token cache is being invalidate because project 96c035e94b9141b98fddc7aba982d815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:38.796820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:38.800597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b6ac40c-bad5-49b5-ab08-ed9695ca4289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:38.806120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 903/1809] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:38 2022] DELETE /identity/v3/projects/96c035e94b9141b98fddc7aba982d815 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:38.809889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:38.810572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:38.811004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:38.866154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:38.866154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:38.972936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 907/1810] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:38.986284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:38.986284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:38.986284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.010708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:39.013142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:39.024668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 904/1811] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:39.035931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:39.046944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441d7f-a393-4f20-aa15-e852b878ede8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:39.063342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:39.066001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.066001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.066001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.066805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:39.067311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:39.069996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:39.098553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 908/1812] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:39.108776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:39.109865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd494f42-aca5-4199-9fdf-0640ac92a237 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:39.115178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:39.115871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.116151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.116487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.117199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:39.119545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:39.309840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 905/1813] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:39 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:39.317181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:39.325546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c30885-0177-48d7-b3f9-e70986c0e884 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:39.417965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:39.419051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.419602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.420092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.421599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:39.429620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:39.634148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 909/1814] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:39 2022] POST /identity/v3/users => generated 287 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:39.641760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:39.682544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e17e5-fee2-4c4a-880f-30bd6cf17579 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:39.695590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:39.697017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.697766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.699697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.700845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:39.703696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:39.755444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 906/1815] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:39.758351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:39.783717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9513-aeb3-4e20-9100-9d5d527f14c2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:39.808351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:39.809921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.811112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.811928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.813383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:39.817273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:39.863325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 910/1816] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:39.865069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:39.890148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ea6df-e57f-428a-bff4-8808b7a78fa1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:39.903433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b74fa-30c8-41f4-840e-4058a66df91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:39.905375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b74fa-30c8-41f4-840e-4058a66df91f admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.906246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b74fa-30c8-41f4-840e-4058a66df91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.907377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b74fa-30c8-41f4-840e-4058a66df91f admin admin] PATH_INFO: `/v3/projects/ed2a724426de48f7b1ffd5dae7b48bb8/users/3f3759c9c8e740f19e0f68a3fe5a27ed/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.939303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b74fa-30c8-41f4-840e-4058a66df91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2a724426de48f7b1ffd5dae7b48bb8, user_id=3f3759c9c8e740f19e0f68a3fe5a27ed, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:39.943043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b74fa-30c8-41f4-840e-4058a66df91f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:40.038326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:40.040854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:40.071493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e2afa-e1ef-49d9-a13d-21fe2c9aa8b3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:40.114088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 907/1817] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:39 2022] PUT /identity/v3/projects/ed2a724426de48f7b1ffd5dae7b48bb8/users/3f3759c9c8e740f19e0f68a3fe5a27ed/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:40.132499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:40.133325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:40.133802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:40.153886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:40.156844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:40.159609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:40.160404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] PATH_INFO: `/v3/users/720baba122af425880e1c22ab482aa69` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:40.170587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=720baba122af425880e1c22ab482aa69)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:40.178072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:40.259245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] MFA Rules not processed for user `3f3759c9c8e740f19e0f68a3fe5a27ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:40.261796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:40.450867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 908/1818] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:40.462890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:40.590533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b4a5-3a9e-4c74-a36c-682b25923b6a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:40.599068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:40.600120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:40.600277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720baba122af425880e1c22ab482aa69'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:40.606497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:40.606497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:40.606916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720baba122af425880e1c22ab482aa69'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:40.607014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:40.607014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:40.620409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720baba122af425880e1c22ab482aa69'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:40.671731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:40.746247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720baba122af425880e1c22ab482aa69'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:40.760177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3b1be99e-8c0b-418d-8d80-1810edc24b98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720baba122af425880e1c22ab482aa69'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:40.766945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 909/1819] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:40 2022] POST /identity/v3/projects => generated 373 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:40.771438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 911/1820] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:40 2022] DELETE /identity/v3/users/720baba122af425880e1c22ab482aa69 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:40.778723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:40.778723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:40.784464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:40.784963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:40.785429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:40.823365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:40.825376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:40.893555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cf29b-14f0-4387-9e07-76c892b59df9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:40.924578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 912/1821] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:40.935004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:40.936806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:40.992057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:40.993247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:40.993908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:40.994755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:40.996043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:41.000386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:41.030327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879aed48-74d8-4df1-bc38-c080e7deb68c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:41.035616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:41.036336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.036623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.036901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.047691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:41.049537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:41.062194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 913/1822] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:41.119948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 910/1823] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:40 2022] POST /identity/v3/users => generated 293 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:41.122021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:41.145123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd45512-a39c-43a3-b7e4-54600582c53c service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:41.148196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:41.149294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.149879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.150502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.152865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:41.156309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:41.177091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 914/1824] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:41.190410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:41.206544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca744e78-b6ad-47e1-8e7b-822eb76e79f1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:41.213780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:41.214525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.218561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.218561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] PATH_INFO: `/v3/projects/6c95ca07df4949cdaee6f99576b24ee9/users/dc42061971e049b08e4b2841f4b63963/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.236156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c95ca07df4949cdaee6f99576b24ee9, user_id=dc42061971e049b08e4b2841f4b63963, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:41.239060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:41.429619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 911/1825] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:41 2022] PUT /identity/v3/projects/6c95ca07df4949cdaee6f99576b24ee9/users/dc42061971e049b08e4b2841f4b63963/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:41.434382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:41.451472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f277c-2a92-4050-8630-9062c4b63aa2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:41.623506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b80300-89f9-4462-814d-f4d0f43ae9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:41.627185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b80300-89f9-4462-814d-f4d0f43ae9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.627766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b80300-89f9-4462-814d-f4d0f43ae9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.628738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b80300-89f9-4462-814d-f4d0f43ae9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.629636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b80300-89f9-4462-814d-f4d0f43ae9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:41.635644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b80300-89f9-4462-814d-f4d0f43ae9fd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:41.662124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 915/1826] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:41.664304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:41.683551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a28a0-2570-4802-8380-5a527affc99c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:41.687930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:41.688624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.688907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.689286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] PATH_INFO: `/v3/projects/6c95ca07df4949cdaee6f99576b24ee9/users/dc42061971e049b08e4b2841f4b63963/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.692267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c95ca07df4949cdaee6f99576b24ee9, user_id=dc42061971e049b08e4b2841f4b63963, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:41.695850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:41.701285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:41.723613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 912/1827] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:41 2022] PUT /identity/v3/projects/6c95ca07df4949cdaee6f99576b24ee9/users/dc42061971e049b08e4b2841f4b63963/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:41.740058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.740668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.741120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.781366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] MFA Rules not processed for user `dc42061971e049b08e4b2841f4b63963`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:41.782802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:41.884542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 916/1828] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:41.897102 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:41.899530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:41.977512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fa991-e433-49ae-9a7e-4fa6b9e0fdf6 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:41.983669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:41.984432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:41.984917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:41.985308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:41.988837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:42.011239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:42.013233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:42.102202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:42.103698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:42.113673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35453-1f68-419a-806a-0981fcbc8f60 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:42.117927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 913/1829] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:42.120111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:42.120782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:42.121073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:42.121356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] PATH_INFO: `/v3/projects/89570a08d3ea428bad8858c7128c3f73` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:42.122780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89570a08d3ea428bad8858c7128c3f73)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:42.124786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:42.340062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89570a08d3ea428bad8858c7128c3f73', 'request_id': 'req-ec722a20-29fc-4126-a257-e70ac37cfc00'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:42.347233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89570a08d3ea428bad8858c7128c3f73', 'request_id': 'req-ec722a20-29fc-4126-a257-e70ac37cfc00'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:42.353981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89570a08d3ea428bad8858c7128c3f73', 'request_id': 'req-ec722a20-29fc-4126-a257-e70ac37cfc00'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:42.479504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] The token cache is being invalidate because project 89570a08d3ea428bad8858c7128c3f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:42.479780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:42.480536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:42.487618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 917/1830] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:42 2022] DELETE /identity/v3/projects/89570a08d3ea428bad8858c7128c3f73 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:42.494046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:42.542733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ca1a3-ec45-48ad-ab55-6b026eb4a355 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:42.644394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:42.645370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:42.645655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:42.645969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] PATH_INFO: `/v3/users/0e3df0a5ca62414889722bad60e7e624` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:42.647273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3df0a5ca62414889722bad60e7e624)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:42.655466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:42.916798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3df0a5ca62414889722bad60e7e624'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:43.004223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3df0a5ca62414889722bad60e7e624'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:43.024228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3df0a5ca62414889722bad60e7e624'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:43.025419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3df0a5ca62414889722bad60e7e624'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:43.039723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b71b2092-8c23-4a1b-b3d1-85ea1c370ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3df0a5ca62414889722bad60e7e624'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:43.052248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 914/1831] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:42 2022] DELETE /identity/v3/users/0e3df0a5ca62414889722bad60e7e624 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:45.182585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:45.184904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:45.211885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec722a20-29fc-4126-a257-e70ac37cfc00 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:45.289635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:45.290328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:45.290612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:45.290897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] PATH_INFO: `/v3/projects/bad242d64e9a44caa7164886f992ff32` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:45.292170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad242d64e9a44caa7164886f992ff32)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:45.294397 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:45.676936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad242d64e9a44caa7164886f992ff32', 'request_id': 'req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:45.677799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad242d64e9a44caa7164886f992ff32', 'request_id': 'req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:45.684483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad242d64e9a44caa7164886f992ff32', 'request_id': 'req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:45.753504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:45.832623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] The token cache is being invalidate because project bad242d64e9a44caa7164886f992ff32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:45.833360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:45.835509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afa6ea4d-acb1-48d0-beb7-69163f13ec8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:45.838324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 918/1832] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:45 2022] DELETE /identity/v3/projects/bad242d64e9a44caa7164886f992ff32 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:45.846737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:45.847367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:45.847680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:45.875345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:45.877179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:45.985110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 915/1833] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:45.999351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:45.999834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.000339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.032931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:46.035502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:46.053805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 919/1834] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:46.060840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.075964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81a858-9651-404a-af7b-c48a0dc2e155 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.099928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.100572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.100935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.101197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.101947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:46.102456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.104034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.125096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 916/1835] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:46.136979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.150909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe024b-835b-454e-a35d-d5f1d622ae97 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.157086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.158893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.159388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.159898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.160746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.162982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.289176 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 920/1836] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:46 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:46.293666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.304626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf0fa-3d0a-44e1-be6a-6e26cd60c4b3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.455830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.456932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.457837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.458236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.459355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.467360 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.550493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:46.640643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 917/1837] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:46 2022] POST /identity/v3/users => generated 293 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:46.648298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.672413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212254a-90dd-4bba-a655-0cbc7e467271 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.683109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.684630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.685116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.685827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.689153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.691529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.720788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 921/1838] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:46.726287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.758305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70986b3f-5b6b-4b86-bd65-bb954cd2eb1a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.770408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fee46a-9019-4b41-a90f-410550d9e60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.771897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee46a-9019-4b41-a90f-410550d9e60b admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.772706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee46a-9019-4b41-a90f-410550d9e60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.774036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee46a-9019-4b41-a90f-410550d9e60b admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.779160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fee46a-9019-4b41-a90f-410550d9e60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.785725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fee46a-9019-4b41-a90f-410550d9e60b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.821507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 918/1839] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:46.837518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.860434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fc473-ae53-4108-8baa-58ddbd6490f7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.870028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.871145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.871620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.872104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] PATH_INFO: `/v3/projects/e01fc25b705d4b08b61a87ec95e7c7b3/users/1013e94e955243819112e770196b5ccb/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.922325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01fc25b705d4b08b61a87ec95e7c7b3, user_id=1013e94e955243819112e770196b5ccb, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.931417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:47.051559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 922/1840] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:46 2022] PUT /identity/v3/projects/e01fc25b705d4b08b61a87ec95e7c7b3/users/1013e94e955243819112e770196b5ccb/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:47.080849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:47.083835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:47.084331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:47.214615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] MFA Rules not processed for user `1013e94e955243819112e770196b5ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:47.217399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:47.417187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 919/1841] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:47.422580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:47.672155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ce8ff-1dc1-41d6-a225-44f5f8ac7a61 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:47.683127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:47.684365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:47.684953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:47.685530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:47.687113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:47.693139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:47.885458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 923/1842] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:47 2022] POST /identity/v3/projects => generated 381 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:47.889328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:47.911331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40f6e-896e-4c08-92ee-cf502e3d6af4 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:48.035494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:48.036138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:48.036424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:48.036706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:48.037352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:48.039371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:48.170172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 920/1843] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:47 2022] POST /identity/v3/users => generated 291 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:48.171110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:48.186957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c7976-a693-4d63-a528-f2e1de1be97a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:48.193206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:48.193881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:48.198084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:48.198084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:48.198084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:48.198084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:48.224924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 924/1844] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:48.227065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:48.253478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea545e-cbdf-4f26-87a7-1435d2876550 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:48.272882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429468f-d758-4270-b69a-17676a3f5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:48.274268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429468f-d758-4270-b69a-17676a3f5840 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:48.274840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429468f-d758-4270-b69a-17676a3f5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:48.275352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429468f-d758-4270-b69a-17676a3f5840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:48.276262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429468f-d758-4270-b69a-17676a3f5840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:48.278556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429468f-d758-4270-b69a-17676a3f5840 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:48.294623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 921/1845] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:48.305391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:48.323064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf501c7-28c7-411f-bda1-c02e80328f90 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:48.337487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:48.338658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:48.340209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:48.341437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] PATH_INFO: `/v3/projects/0e7bfd3df2cb43ffb3f15cce594c59cc/users/b0cbc2c7b2be409ebe096b0c046e41b7/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:48.387259 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7bfd3df2cb43ffb3f15cce594c59cc, user_id=b0cbc2c7b2be409ebe096b0c046e41b7, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:48.397316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:48.594141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 925/1846] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:48 2022] PUT /identity/v3/projects/0e7bfd3df2cb43ffb3f15cce594c59cc/users/b0cbc2c7b2be409ebe096b0c046e41b7/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:48.613667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:48.614444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:48.614978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:48.648946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] MFA Rules not processed for user `b0cbc2c7b2be409ebe096b0c046e41b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:48.650305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:48.753653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 922/1847] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:48.758369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:48.838660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:48.844005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571f88-dcc0-49a0-9fa4-b34a756c0b6f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:48.850608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:48.851337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:48.851744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:48.852084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:48.852900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:48.855262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:49.108090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 926/1848] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:49.113673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:49.132198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225dc83-fecc-4b5c-8105-6ca3a0b1895d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:49.278701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:49.281369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:49.282287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:49.283257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:49.284835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:49.291814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:49.480831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:49.482899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:49.498993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22b410-ab94-41d3-8dac-1a9fe2195093 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:49.501244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 923/1849] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:49 2022] POST /identity/v3/users => generated 298 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:49.508007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:49.509248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:49.509681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:49.510207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] PATH_INFO: `/v3/users/3f3759c9c8e740f19e0f68a3fe5a27ed` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:49.510856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:49.511698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3759c9c8e740f19e0f68a3fe5a27ed)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:49.513694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:49.529503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4617-9da9-4040-99ff-243668115646 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:49.559297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:49.561619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:49.561619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:49.561619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:49.567142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:49.572945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:49.616687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 924/1850] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:49.617787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:49.645713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4422-6a9f-4a9c-a2b9-34b18dc1a3cb admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:49.660967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:49.664128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:49.665685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:49.665685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] PATH_INFO: `/v3/projects/99082a3be3f64d2c938e04330fa7f7c0/users/aab48e6052734bb4832f34846772a968/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:49.728058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99082a3be3f64d2c938e04330fa7f7c0, user_id=aab48e6052734bb4832f34846772a968, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:49.733725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:49.827698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3759c9c8e740f19e0f68a3fe5a27ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:49.828862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3759c9c8e740f19e0f68a3fe5a27ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:49.843879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:49.847289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3759c9c8e740f19e0f68a3fe5a27ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:49.857188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 925/1851] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:49 2022] PUT /identity/v3/projects/99082a3be3f64d2c938e04330fa7f7c0/users/aab48e6052734bb4832f34846772a968/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:49.862286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:49.940169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc03c-5559-4723-8288-0991a27a6299 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:49.945056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:49.945692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:49.946013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:49.946289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:49.947612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:49.949979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:49.968140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 926/1852] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:06:49.968459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3759c9c8e740f19e0f68a3fe5a27ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:49.969356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:49.976036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c8aeafd-57e6-4c08-9355-75fec9cf355f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3759c9c8e740f19e0f68a3fe5a27ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:49.981198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a53f5-98c7-4af5-b7a6-c5998438d48a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:49.983125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 927/1853] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:49 2022] DELETE /identity/v3/users/3f3759c9c8e740f19e0f68a3fe5a27ed => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:49.988838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:49.989168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:49.989323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:50.010275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:50.011552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:50.054552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa969e-546a-436f-ab2f-cbe555d112a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:50.055242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa969e-546a-436f-ab2f-cbe555d112a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:50.055524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa969e-546a-436f-ab2f-cbe555d112a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:50.055804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa969e-546a-436f-ab2f-cbe555d112a7 admin admin] PATH_INFO: `/v3/projects/99082a3be3f64d2c938e04330fa7f7c0/users/aab48e6052734bb4832f34846772a968/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:50.058707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa969e-546a-436f-ab2f-cbe555d112a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99082a3be3f64d2c938e04330fa7f7c0, user_id=aab48e6052734bb4832f34846772a968, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:50.063795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa969e-546a-436f-ab2f-cbe555d112a7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:50.083339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 927/1854] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:49 2022] PUT /identity/v3/projects/99082a3be3f64d2c938e04330fa7f7c0/users/aab48e6052734bb4832f34846772a968/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:50.093836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 928/1855] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:50.098664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:50.098958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:50.099242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:50.104703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:50.107166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:50.145057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] MFA Rules not processed for user `aab48e6052734bb4832f34846772a968`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:50.148301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:50.195846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a292dba-78f2-4337-9be9-fa79009e0ec4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:50.247433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 928/1856] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:50.266054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:50.267751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:50.279198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:50.280906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:50.281557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:50.282247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:50.364540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126e2f6-ba00-4e4d-a0ef-c3ae679b5ce6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:50.370835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:50.371997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:50.371997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:50.372302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:50.374402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:50.400793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:50.402324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:50.415826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 929/1857] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:50.467981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:50.473426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:50.496963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 929/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:51.206206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:51.209076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:51.228339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a6e45-3a41-479f-a71e-c09392c8de6c service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:51.237824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:51.238312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:51.239320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:51.239793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:51.240221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] PATH_INFO: `/v3/projects/ed2a724426de48f7b1ffd5dae7b48bb8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:51.241971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2a724426de48f7b1ffd5dae7b48bb8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:51.245503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:51.369570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:51.495807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2a724426de48f7b1ffd5dae7b48bb8', 'request_id': 'req-58fc5f29-3c52-4089-bf67-e8201e41f136'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:51.506450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2a724426de48f7b1ffd5dae7b48bb8', 'request_id': 'req-58fc5f29-3c52-4089-bf67-e8201e41f136'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:51.515790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2a724426de48f7b1ffd5dae7b48bb8', 'request_id': 'req-58fc5f29-3c52-4089-bf67-e8201e41f136'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:51.641267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] The token cache is being invalidate because project ed2a724426de48f7b1ffd5dae7b48bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:51.641593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:51.642547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:51.648058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 930/1859] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:51 2022] DELETE /identity/v3/projects/ed2a724426de48f7b1ffd5dae7b48bb8 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:51.651452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:51.672182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed64d6-e2ee-499f-9474-945fe2e88adb service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:51.804237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:51.804956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:51.805309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:51.805665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] PATH_INFO: `/v3/users/dc42061971e049b08e4b2841f4b63963` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:51.808407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc42061971e049b08e4b2841f4b63963)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:51.810960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:52.210113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc42061971e049b08e4b2841f4b63963'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:52.296440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:52.318231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc42061971e049b08e4b2841f4b63963'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:52.363660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc42061971e049b08e4b2841f4b63963'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:52.364963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc42061971e049b08e4b2841f4b63963'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:52.381791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc42061971e049b08e4b2841f4b63963'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:52.395736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 930/1860] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:51 2022] DELETE /identity/v3/users/dc42061971e049b08e4b2841f4b63963 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:53.198545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:53.259414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:53.261722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:53.284895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5f29-3c52-4089-bf67-e8201e41f136 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:53.404278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:53.407595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:53.407595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:53.407595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] PATH_INFO: `/v3/users/1dc0cdb78a5f4db3892135af0507668c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:53.407595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dc0cdb78a5f4db3892135af0507668c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:53.411152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:53.630414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:53.632345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:53.841789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dc0cdb78a5f4db3892135af0507668c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:53.842826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3df0a-a67c-43f0-96d6-7dc52d0aa379 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:53.843168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dc0cdb78a5f4db3892135af0507668c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:53.853438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:53.854131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:53.854419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:53.854703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:53.856703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:53.874358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dc0cdb78a5f4db3892135af0507668c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:53.951597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:53.952637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:53.982654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 931/1861] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:53.984260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:53.997650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.005794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dc0cdb78a5f4db3892135af0507668c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.026975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dc0cdb78a5f4db3892135af0507668c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.051659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c6ba0-5693-4127-8897-41f0644ae67c service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:54.066980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 931/1862] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:53 2022] DELETE /identity/v3/users/1dc0cdb78a5f4db3892135af0507668c => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:54.068063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:54.071289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.076204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:54.077362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.077863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.083483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] PATH_INFO: `/v3/projects/6c95ca07df4949cdaee6f99576b24ee9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.086508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c95ca07df4949cdaee6f99576b24ee9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:54.095022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:54.172064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae665-16b5-4df6-86ec-5f7f095c8446 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:54.181028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:54.182179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.182812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.183249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] PATH_INFO: `/v3/users/2ca0bd2bc636418daf59075a644f0bfd` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.186232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca0bd2bc636418daf59075a644f0bfd)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:54.188503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:54.389065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c95ca07df4949cdaee6f99576b24ee9', 'request_id': 'req-f248c4aa-9891-40b8-87ff-9be88a9ebab6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.408432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c95ca07df4949cdaee6f99576b24ee9', 'request_id': 'req-f248c4aa-9891-40b8-87ff-9be88a9ebab6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.502262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c95ca07df4949cdaee6f99576b24ee9', 'request_id': 'req-f248c4aa-9891-40b8-87ff-9be88a9ebab6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.521023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca0bd2bc636418daf59075a644f0bfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.522869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca0bd2bc636418daf59075a644f0bfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.524770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] The token cache is being invalidate because project 6c95ca07df4949cdaee6f99576b24ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:54.525290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.527545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f248c4aa-9891-40b8-87ff-9be88a9ebab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.531965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 932/1863] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:53 2022] DELETE /identity/v3/projects/6c95ca07df4949cdaee6f99576b24ee9 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:54.535821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a5756-fd92-4cdc-a792-bb2186f06602 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.536552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a5756-fd92-4cdc-a792-bb2186f06602 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.540259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a5756-fd92-4cdc-a792-bb2186f06602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.549139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca0bd2bc636418daf59075a644f0bfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.575659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-741a5756-fd92-4cdc-a792-bb2186f06602 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:54.577675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-741a5756-fd92-4cdc-a792-bb2186f06602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.656078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca0bd2bc636418daf59075a644f0bfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.670839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be222b5a-d58c-4a4e-b0d1-8d6a5603cc6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca0bd2bc636418daf59075a644f0bfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:54.680610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 932/1864] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:54 2022] DELETE /identity/v3/users/2ca0bd2bc636418daf59075a644f0bfd => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:54.684013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.686947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 933/1865] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:54.688523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5027de-bc06-4b62-9919-1b5ef012510e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.688998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5027de-bc06-4b62-9919-1b5ef012510e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.690081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5027de-bc06-4b62-9919-1b5ef012510e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.691348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.691965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.717945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:54.719050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1e5027de-bc06-4b62-9919-1b5ef012510e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:54.719308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.722151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1e5027de-bc06-4b62-9919-1b5ef012510e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.833625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 933/1866] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:54.833625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:54.833625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.837642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 934/1867] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:54.839941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.842553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.842553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.872079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:54.873746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.883817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 935/1868] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:54.884528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:54.886132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:54.956553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c1d75-40cb-4f24-8d34-e39289289faa None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:54.967320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:54.968626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:54.969169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:54.970547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:54.996994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:54.999920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.015693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911108-82fd-4f1b-b99b-17f899bc78f1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.022382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.022904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 934/1869] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:55.024629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:55.037872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.038773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.039163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.045359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f323195-e3d2-44ed-a1fc-a3b037d36b8a service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.060938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.062748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.063402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.063790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.064556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.065922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.068033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:55.068033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.071011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.079679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 936/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:55.103111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 935/1871] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:06:55.103952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:55.123535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb68164-e1ee-4c4e-a36a-a45a8dde605f service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.136412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.137618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.138182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.138702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.140581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.146163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.325046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 937/1872] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:55 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:06:55.326350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:55.391535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9536cb-5db6-480a-bc1f-c585635b134d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.478415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:55.583626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.584607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.584607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.585058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.585812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.588477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.763048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 936/1873] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:55 2022] POST /identity/v3/users => generated 288 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:55.763737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:55.790156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203edc-cbd4-4f11-af87-6c6f9387fe50 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.825670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.825670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.825670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.825670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.825997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.825997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.831826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 938/1874] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:55.837554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:55.852597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba7b54-e458-4ab2-aea8-8f4f8d60214c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.868061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b11163-8592-4218-acf9-028fb0243f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.871339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b11163-8592-4218-acf9-028fb0243f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.871339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b11163-8592-4218-acf9-028fb0243f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.871339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b11163-8592-4218-acf9-028fb0243f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.874613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b11163-8592-4218-acf9-028fb0243f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.876798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b11163-8592-4218-acf9-028fb0243f6b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:55.894529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 937/1875] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:06:55.895078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:55.912378 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec34f8c-01ed-4f9d-b3a7-7509bcd30096 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:55.923279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:55.924119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:55.924936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:55.932570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] PATH_INFO: `/v3/projects/6c1caba92ba04650adb1f0ad7615b7a2/users/16cfdd172dbc41b795a82c97185229e4/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:55.977738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1caba92ba04650adb1f0ad7615b7a2, user_id=16cfdd172dbc41b795a82c97185229e4, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:55.984380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:56.099651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 939/1876] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:06:55 2022] PUT /identity/v3/projects/6c1caba92ba04650adb1f0ad7615b7a2/users/16cfdd172dbc41b795a82c97185229e4/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:56.103169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:56.103592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:56.103972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:56.137238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] MFA Rules not processed for user `16cfdd172dbc41b795a82c97185229e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:56.138421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:56.245347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 938/1877] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:56.271384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:56.273501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:56.283699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:56.289383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:56.443180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e390fd-8e10-4108-a56d-ae8fe7d94abe admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:56.448703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:56.449497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:56.449888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:56.450556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:56.464009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:56.465805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:56.493126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209dfd3-5049-4f7d-8634-7133100a27cb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:56.496792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 940/1878] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:56.498847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:56.502042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:56.505668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:56.507415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:56.508511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:56.509211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] PATH_INFO: `/v3/projects/02d0ef28230e4e938d59bc2b389f0c1b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:56.512101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d0ef28230e4e938d59bc2b389f0c1b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:56.513402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:56.518115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:56.531496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad4219-fe06-4f5b-9879-56828d121b53 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:56.539791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:56.540638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:56.541092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:56.541535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] PATH_INFO: `/v3/projects/c457ca6329364295baa6922524acb58d` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:56.543389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c457ca6329364295baa6922524acb58d)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:56.546620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:56.774691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c457ca6329364295baa6922524acb58d', 'request_id': 'req-d937032f-5bc1-4864-8594-09a20b1156ba'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.779461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d0ef28230e4e938d59bc2b389f0c1b', 'request_id': 'req-175be6b9-8fac-436c-ab45-f44afda11568'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.794129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c457ca6329364295baa6922524acb58d', 'request_id': 'req-d937032f-5bc1-4864-8594-09a20b1156ba'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.807889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d0ef28230e4e938d59bc2b389f0c1b', 'request_id': 'req-175be6b9-8fac-436c-ab45-f44afda11568'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.819526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d0ef28230e4e938d59bc2b389f0c1b', 'request_id': 'req-175be6b9-8fac-436c-ab45-f44afda11568'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.868062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c457ca6329364295baa6922524acb58d', 'request_id': 'req-d937032f-5bc1-4864-8594-09a20b1156ba'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.893621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] The token cache is being invalidate because project c457ca6329364295baa6922524acb58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:56.894705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.896026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.904184 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 941/1879] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:56 2022] DELETE /identity/v3/projects/c457ca6329364295baa6922524acb58d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:56.909922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:56.912252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:56.962057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] The token cache is being invalidate because project 02d0ef28230e4e938d59bc2b389f0c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:56.962498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.963330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:56.972236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 939/1880] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:56 2022] DELETE /identity/v3/projects/02d0ef28230e4e938d59bc2b389f0c1b => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:56.974974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:56.979289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:57.069233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937032f-5bc1-4864-8594-09a20b1156ba admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:57.074848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:57.075628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:57.075911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:57.076291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] PATH_INFO: `/v3/users/6f5acd2942d340b898873e536dc55604` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:57.077746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5acd2942d340b898873e536dc55604)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:57.080049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:57.147341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be6b9-8fac-436c-ab45-f44afda11568 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:57.155626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:57.156318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:57.156639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:57.156917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] PATH_INFO: `/v3/users/aa2ed3261fbd4d3f8f034f2edca1dbe7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:57.158337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2ed3261fbd4d3f8f034f2edca1dbe7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:57.160641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:57.361996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5acd2942d340b898873e536dc55604'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.441264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2ed3261fbd4d3f8f034f2edca1dbe7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.442224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2ed3261fbd4d3f8f034f2edca1dbe7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.458921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2ed3261fbd4d3f8f034f2edca1dbe7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.679319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5acd2942d340b898873e536dc55604'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.701915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5acd2942d340b898873e536dc55604'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.703275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5acd2942d340b898873e536dc55604'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.725453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5acd2942d340b898873e536dc55604'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.740014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 942/1881] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:56 2022] DELETE /identity/v3/users/6f5acd2942d340b898873e536dc55604 => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:06:57.746321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:57.748849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:57.755602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2ed3261fbd4d3f8f034f2edca1dbe7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.793548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2ed3261fbd4d3f8f034f2edca1dbe7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:57.812830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 940/1882] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:56 2022] DELETE /identity/v3/users/aa2ed3261fbd4d3f8f034f2edca1dbe7 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:57.885720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c1d1-14d3-4bed-a85f-056355346e60 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:57.979656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:57.980310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:57.981523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:57.981841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:57.984818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:58.107890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:58.109453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:58.123410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 943/1883] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:58.324917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:58.408620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:58.641707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:58.645111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:58.827580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baa840-9087-4300-85b1-b17e684ef16a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:58.827580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:58.827580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:58.827580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:58.828451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] PATH_INFO: `/v3/projects/59fc2b9959c04130838f48189657249f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:58.828451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59fc2b9959c04130838f48189657249f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:58.832121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:58.935191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:58.938098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:58.959217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b9dc-9600-46f1-9414-674994fc9ccd service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:58.966103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:58.967244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:58.967709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:58.968194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] PATH_INFO: `/v3/projects/056fdf4d6eae49809b29e4a9268abe90` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:58.971230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=056fdf4d6eae49809b29e4a9268abe90)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:58.974258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:59.001281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59fc2b9959c04130838f48189657249f', 'request_id': 'req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.013862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59fc2b9959c04130838f48189657249f', 'request_id': 'req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.022109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59fc2b9959c04130838f48189657249f', 'request_id': 'req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.116289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056fdf4d6eae49809b29e4a9268abe90', 'request_id': 'req-d3a9d826-4eb2-4673-a69c-49512335a2cc'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.125222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056fdf4d6eae49809b29e4a9268abe90', 'request_id': 'req-d3a9d826-4eb2-4673-a69c-49512335a2cc'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.131854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] The token cache is being invalidate because project 59fc2b9959c04130838f48189657249f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:59.132136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.133066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6ede44d6-88b0-4fae-8db9-be9f3d09a61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.134723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 941/1884] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:58 2022] DELETE /identity/v3/projects/59fc2b9959c04130838f48189657249f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:59.139770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1f3c-ecd8-4a76-9432-e8cadf5201b0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.140773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1f3c-ecd8-4a76-9432-e8cadf5201b0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.141130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1f3c-ecd8-4a76-9432-e8cadf5201b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.165633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-421e1f3c-ecd8-4a76-9432-e8cadf5201b0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:59.167720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-421e1f3c-ecd8-4a76-9432-e8cadf5201b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:59.227465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056fdf4d6eae49809b29e4a9268abe90', 'request_id': 'req-d3a9d826-4eb2-4673-a69c-49512335a2cc'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.265875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] The token cache is being invalidate because project 056fdf4d6eae49809b29e4a9268abe90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:59.269270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.270415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d3a9d826-4eb2-4673-a69c-49512335a2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:59.273701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 944/1885] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:58 2022] DELETE /identity/v3/projects/056fdf4d6eae49809b29e4a9268abe90 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:59.283538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8994-8f26-4b29-acfe-fcae1cb14005 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.283928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8994-8f26-4b29-acfe-fcae1cb14005 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.284298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8994-8f26-4b29-acfe-fcae1cb14005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.340775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 942/1886] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:59.352035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.352965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.353402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.368399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6dad8994-8f26-4b29-acfe-fcae1cb14005 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:59.371387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6dad8994-8f26-4b29-acfe-fcae1cb14005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:59.380370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:59.382189 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:59.389951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 945/1887] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:06:59.395757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 943/1888] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:59.400636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:59.400830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.401196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.401685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.412626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3291042-c194-4043-86d1-905a7c8be6ba None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:59.418180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:59.419103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.419472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.419848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.420670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:59.421151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:59.423410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:59.436820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:59.438808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:59.445453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 944/1889] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:06:59.446742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:59.461278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 946/1890] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:59.461726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67cca-a929-4513-8364-ffd64db24064 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:59.465525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:59.468357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:59.469251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.469613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.470032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.471105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:59.473523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:59.480117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af72763-7999-483c-944e-06a59c34ab81 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:59.491394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:59.492256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.492757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.493273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.494397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:06:59.494979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:59.497082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:59.513621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 947/1891] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:06:59.521316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:59.534794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4b857-5591-4762-b7cb-8dfff1dcccab admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:59.542363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:59.543478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.544034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.544997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.546344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:59.549887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:59.617265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 945/1892] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:59.618703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:59.743592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 948/1893] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/projects => generated 401 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:06:59.744798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:59.749753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846c3e0-5d15-4884-a4aa-72e3c319ea8d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:59.906778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05a74e-d6e9-484f-811e-2122126e8c85 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:59.916984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:59.918130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.918890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.919682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.920704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:59.928114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:59.933769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:59.935004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:59.935528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:59.936105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:59.937384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:59.946106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.133425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 946/1894] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/users => generated 285 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:00.134149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 949/1895] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:06:59 2022] POST /identity/v3/users => generated 301 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:00.138674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.138888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.166612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea96c9-7ca1-4b3e-8407-d5d0b470235a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.171945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.172600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.172914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.173194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.174714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.176383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.193476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 950/1896] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:00.193674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7d107-a748-4bd6-83ad-5dad1a28da63 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.194724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.203573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.205495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.205975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.206486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.207233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.208325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c74ee-2185-4401-be63-54acecfb45c2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.208879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.213432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.214122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.214402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.214679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.215332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.217072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.221976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 947/1897] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:00.226371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.242297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc24adb-a8db-4202-b53c-38361536b326 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.250168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 951/1898] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:00.251201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.253723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.254539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.254831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.255156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.255839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.257949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.263174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053a8f-db18-434a-8978-f0afb9031b26 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.269170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c60c3-b35c-402a-b141-99fc44dd5740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.269978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c60c3-b35c-402a-b141-99fc44dd5740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.270277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c60c3-b35c-402a-b141-99fc44dd5740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.270553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c60c3-b35c-402a-b141-99fc44dd5740 admin admin] PATH_INFO: `/v3/projects/c0024c9db04e416e8f473af7b1ba0e83/users/8c64624b07c047dd972efaf953f66b3e/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.274256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 948/1899] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:07:00.279900 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.289353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c60c3-b35c-402a-b141-99fc44dd5740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0024c9db04e416e8f473af7b1ba0e83, user_id=8c64624b07c047dd972efaf953f66b3e, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.292174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c60c3-b35c-402a-b141-99fc44dd5740 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.293723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44376b80-7f6e-4d03-a889-b7884132498c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.300204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce058fd-1bfc-48ef-bd5d-69a9097930ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.300921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce058fd-1bfc-48ef-bd5d-69a9097930ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.301214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce058fd-1bfc-48ef-bd5d-69a9097930ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.301499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce058fd-1bfc-48ef-bd5d-69a9097930ab admin admin] PATH_INFO: `/v3/projects/1d91ac516fff4c7f9933249775113c93/users/2b8cd9f063f344349332b216165628bf/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.327267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce058fd-1bfc-48ef-bd5d-69a9097930ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d91ac516fff4c7f9933249775113c93, user_id=2b8cd9f063f344349332b216165628bf, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.330876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce058fd-1bfc-48ef-bd5d-69a9097930ab admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.443073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 952/1900] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:00 2022] PUT /identity/v3/projects/c0024c9db04e416e8f473af7b1ba0e83/users/8c64624b07c047dd972efaf953f66b3e/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:00.444831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 949/1901] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:00 2022] PUT /identity/v3/projects/1d91ac516fff4c7f9933249775113c93/users/2b8cd9f063f344349332b216165628bf/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:00.461642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.462161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.462554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.476759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.476759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.476759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.516665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] MFA Rules not processed for user `8c64624b07c047dd972efaf953f66b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:00.519397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:00.526768 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] MFA Rules not processed for user `2b8cd9f063f344349332b216165628bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:00.529265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:00.627515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 953/1902] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:00.632120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.671903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 950/1903] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:00.678913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.712555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:00.757872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb586d4-e663-4bb6-b452-c707e1bd6e69 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.776740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.778447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.779314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.780061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.782117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.790085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.799869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b6e6-01dc-48d8-bd01-d27a91da3e5a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:00.806823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:00.807549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:00.807846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:00.808132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:00.808793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:00.810986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:00.933076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 954/1904] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:00 2022] POST /identity/v3/projects => generated 399 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:00.936715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 951/1905] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:00 2022] POST /identity/v3/projects => generated 369 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:00.937932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:00.938098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.024461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045747e7-a5a6-43d4-97e5-a44cc4ca4beb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.037060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82b531-5d5c-427e-b72b-0dc9fc54bf96 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.043171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.043942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.044333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.044864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.045933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.052252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.052400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.052400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.052400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.052400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.052962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.054694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.182495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 955/1906] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:00 2022] POST /identity/v3/users => generated 291 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:01.185115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.189874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 952/1907] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:00 2022] POST /identity/v3/users => generated 306 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:01.195339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.200437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867685f3-8632-45ff-a85a-3154f4076154 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.207062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.207951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.208182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.208674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.209226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.213863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.214167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003013c-884f-4ef5-82d9-d3655029539d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.218737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.219670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.220069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.220847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.221869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.224104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.228813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 956/1908] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:01.234004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.247083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 953/1909] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:01.250420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.251446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc8886-450c-4942-b44f-0deea6348b43 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.257151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.257851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.258178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.258456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] PATH_INFO: `/v3/projects/4c311fec58c541df9cc1c86e1c1321e8/users/0bfadcf3637a4e698e271b056212d46b/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.268363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59771ee2-2611-4ebe-8628-5cb82c917914 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.275891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.276660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.277063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.277421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] PATH_INFO: `/v3/projects/2447d6122b314ee293bf4cbbf4381a1b/users/e84c60a210a6401eae47850954226cf1/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.285596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c311fec58c541df9cc1c86e1c1321e8, user_id=0bfadcf3637a4e698e271b056212d46b, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.290496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.296739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2447d6122b314ee293bf4cbbf4381a1b, user_id=e84c60a210a6401eae47850954226cf1, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.299723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.457041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 954/1910] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:01 2022] PUT /identity/v3/projects/2447d6122b314ee293bf4cbbf4381a1b/users/e84c60a210a6401eae47850954226cf1/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:01.463214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.470615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 957/1911] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:01 2022] PUT /identity/v3/projects/4c311fec58c541df9cc1c86e1c1321e8/users/0bfadcf3637a4e698e271b056212d46b/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:01.476473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.547669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9b26-47f5-4c80-b261-436dae197db6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.554149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.554801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.555083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.555365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.556032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.557740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.558333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a14f4-e27e-45d2-8b0d-0d4361194fe5 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.563431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.564074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.565056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.565316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.566021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.567872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.569563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 955/1912] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:01.574131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.586304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68ff98-6a3b-4b30-8e28-fdd60c70bc00 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.591299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 958/1913] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:01.591558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255c21b-96c6-49fb-8078-3e7a46719175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.592129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255c21b-96c6-49fb-8078-3e7a46719175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.592416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255c21b-96c6-49fb-8078-3e7a46719175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.592698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255c21b-96c6-49fb-8078-3e7a46719175 admin admin] PATH_INFO: `/v3/projects/2447d6122b314ee293bf4cbbf4381a1b/users/e84c60a210a6401eae47850954226cf1/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.594930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255c21b-96c6-49fb-8078-3e7a46719175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2447d6122b314ee293bf4cbbf4381a1b, user_id=e84c60a210a6401eae47850954226cf1, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.596776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.598640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255c21b-96c6-49fb-8078-3e7a46719175 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.610556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e261141-6e65-4b8a-9fac-2bec9d56837c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.616703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 956/1914] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:01 2022] PUT /identity/v3/projects/2447d6122b314ee293bf4cbbf4381a1b/users/e84c60a210a6401eae47850954226cf1/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:01.626370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.627813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.627813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.661869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] MFA Rules not processed for user `e84c60a210a6401eae47850954226cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:01.663160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:01.720263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2728d3-48e0-4c2f-a52f-6288585fc1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.721788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2728d3-48e0-4c2f-a52f-6288585fc1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.722343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2728d3-48e0-4c2f-a52f-6288585fc1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.722748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2728d3-48e0-4c2f-a52f-6288585fc1d7 admin admin] PATH_INFO: `/v3/projects/4c311fec58c541df9cc1c86e1c1321e8/users/0bfadcf3637a4e698e271b056212d46b/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.725172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2728d3-48e0-4c2f-a52f-6288585fc1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c311fec58c541df9cc1c86e1c1321e8, user_id=0bfadcf3637a4e698e271b056212d46b, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:01.729681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2728d3-48e0-4c2f-a52f-6288585fc1d7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:01.747612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 959/1915] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:01 2022] PUT /identity/v3/projects/4c311fec58c541df9cc1c86e1c1321e8/users/0bfadcf3637a4e698e271b056212d46b/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:01.766794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.769025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.769676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:01.779942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 957/1916] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:01.793588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:01.795706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:01.815319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] MFA Rules not processed for user `0bfadcf3637a4e698e271b056212d46b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:01.815319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:01.883660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12307a-b2e2-4124-9422-a0d116e8fc24 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:01.962944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 960/1917] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:01.987236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:01.990508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:01.990508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:01.990508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:02.030929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:02.034126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:02.053668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136531e0-5a25-44a8-8d9e-2a68a8bfbfab None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:02.057927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:02.058748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:02.059064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:02.059590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:02.071130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:02.072515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:02.084937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:02.087241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 961/1918] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:02.104053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:02.105558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:02.119577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 958/1919] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:02.315209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:02.318574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:02.406654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b635ac-2b0a-4751-9439-6624c0bdb820 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:02.414598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:02.415676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:02.416229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:02.416704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] PATH_INFO: `/v3/users/16cfdd172dbc41b795a82c97185229e4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:02.418887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=16cfdd172dbc41b795a82c97185229e4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:02.422741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:02.718402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:02.727972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2aa8-ffa2-4bab-97df-54ba890ec07f service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:02.732200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cc10c-f4ce-4006-a4d3-733afcbc4f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:02.732993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cc10c-f4ce-4006-a4d3-733afcbc4f5b service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:02.733288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cc10c-f4ce-4006-a4d3-733afcbc4f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:02.733576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cc10c-f4ce-4006-a4d3-733afcbc4f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:02.789612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:02.791792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16cfdd172dbc41b795a82c97185229e4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:02.811037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cc10c-f4ce-4006-a4d3-733afcbc4f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:02.812567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cc10c-f4ce-4006-a4d3-733afcbc4f5b service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:02.835691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 959/1920] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:02.964051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16cfdd172dbc41b795a82c97185229e4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:02.981140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16cfdd172dbc41b795a82c97185229e4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:02.982320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16cfdd172dbc41b795a82c97185229e4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:02.990615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16cfdd172dbc41b795a82c97185229e4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:02.997517 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 962/1921] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:02 2022] DELETE /identity/v3/users/16cfdd172dbc41b795a82c97185229e4 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:03.003413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:03.003902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:03.004106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:03.027145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:03.028420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:03.096792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 960/1922] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:03.107707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:03.109265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:03.185327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ba3e5-e94b-48b2-91ad-bb33021543ef admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:03.191092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d277db-6276-42e0-abfe-be48f2ecde44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:03.191938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d277db-6276-42e0-abfe-be48f2ecde44 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:03.192404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d277db-6276-42e0-abfe-be48f2ecde44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:03.192801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d277db-6276-42e0-abfe-be48f2ecde44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:03.204233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d277db-6276-42e0-abfe-be48f2ecde44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:03.206731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d277db-6276-42e0-abfe-be48f2ecde44 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:03.221135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 963/1923] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:03.862740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:03.865732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:03.903917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72298c-6755-4f40-a7f2-f5c5a66c660c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:03.921178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:03.927381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:03.930717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:03.931325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] PATH_INFO: `/v3/projects/6c1caba92ba04650adb1f0ad7615b7a2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:03.935321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1caba92ba04650adb1f0ad7615b7a2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:03.941579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:04.328024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1caba92ba04650adb1f0ad7615b7a2', 'request_id': 'req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:04.353699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1caba92ba04650adb1f0ad7615b7a2', 'request_id': 'req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:04.403984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1caba92ba04650adb1f0ad7615b7a2', 'request_id': 'req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:04.550146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] The token cache is being invalidate because project 6c1caba92ba04650adb1f0ad7615b7a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:04.550875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:04.552490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-be0ec357-3fd9-4fe3-8660-1398fb96f7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:04.556370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 961/1924] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:03 2022] DELETE /identity/v3/projects/6c1caba92ba04650adb1f0ad7615b7a2 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:04.563138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:04.563854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:04.563854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:04.592528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:04.594033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:04.683310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 964/1925] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:04.691947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:04.692586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:04.693103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:04.723806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:04.725834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:04.740052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 962/1926] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:04.747714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:04.761569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4457ec4-4ad7-4a14-83be-6626d2ee4a8a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:04.769505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:04.770652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:04.771143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:04.771681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:04.772713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:04.773304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:04.775632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:04.800355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 965/1927] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:07:04.805754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:04.819547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0dc32-492a-4c3f-a416-559407f47389 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:04.833618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:04.834737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:04.835213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:04.835692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:04.836877 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:04.846301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:04.991730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 963/1928] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:04 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:04.996616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:05.009227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fed1b-73be-4d7c-a44a-f44cfae99269 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:05.092008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:05.093480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.093996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.096741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.096741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:05.098014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:05.190954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:05.200554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 966/1929] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:04 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:05.206741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:05.219285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb317-58ac-4428-96f9-207cd70bfeb3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:05.225036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:05.225679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.226005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.226290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.226956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:05.228998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:05.245201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 964/1930] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:05.249396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:05.261603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3a807-27c7-44bb-ae70-1d87bfb015b2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:05.266666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8304-37cd-4e5d-bef4-15a483688195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:05.267292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8304-37cd-4e5d-bef4-15a483688195 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.268156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8304-37cd-4e5d-bef4-15a483688195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.268619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8304-37cd-4e5d-bef4-15a483688195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.269357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b8304-37cd-4e5d-bef4-15a483688195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:05.271187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b8304-37cd-4e5d-bef4-15a483688195 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:05.283983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 967/1931] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:05.288492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:05.301464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fdaa1-1a14-4e68-94a5-449afc64879a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:05.306529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:05.307226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.307511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.307792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] PATH_INFO: `/v3/projects/37239e3218ed49a69f17e3231ceabbcf/users/c189a6afc7564155944552831350b979/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.331014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37239e3218ed49a69f17e3231ceabbcf, user_id=c189a6afc7564155944552831350b979, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:05.336705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:05.496569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 965/1932] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:05 2022] PUT /identity/v3/projects/37239e3218ed49a69f17e3231ceabbcf/users/c189a6afc7564155944552831350b979/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:05.518931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.519437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.519868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.594756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] MFA Rules not processed for user `c189a6afc7564155944552831350b979`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:05.597569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:05.703869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:05.710926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 968/1933] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:05.738990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:05.746251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:05.768764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:05.771271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:05.842849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6cf0-6c51-46aa-9c74-c7c09bc3a66e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:05.847595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:05.848266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.848554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.848837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] PATH_INFO: `/v3/users/1013e94e955243819112e770196b5ccb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.850409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1013e94e955243819112e770196b5ccb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:05.852375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:05.858005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf9ca6-fdbe-48d3-ba27-101fa1f69348 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:05.862792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e368-cfb9-4f0e-8539-e6993c48a083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:05.863693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e368-cfb9-4f0e-8539-e6993c48a083 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:05.865571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e368-cfb9-4f0e-8539-e6993c48a083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:05.865571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e368-cfb9-4f0e-8539-e6993c48a083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:05.875643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11e368-cfb9-4f0e-8539-e6993c48a083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:05.877834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11e368-cfb9-4f0e-8539-e6993c48a083 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:05.888922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 969/1934] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:06.180830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1013e94e955243819112e770196b5ccb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:06.181517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1013e94e955243819112e770196b5ccb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:06.195107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1013e94e955243819112e770196b5ccb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:06.289507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1013e94e955243819112e770196b5ccb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:06.299690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1013e94e955243819112e770196b5ccb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:06.308524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 966/1935] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:05 2022] DELETE /identity/v3/users/1013e94e955243819112e770196b5ccb => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:06.313111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:06.313397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:06.313675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:06.336105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:06.337359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:06.415428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 970/1936] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:06.426172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:06.427692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:06.522312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b26ef8-9d4f-4295-b37b-9efc46367b12 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:06.526348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:06.526980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:06.527262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:06.527535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:06.538769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:06.540447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:06.553138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 967/1937] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:07.141494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:07.286882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:07.291119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:07.313403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d061ba-d126-4460-9df0-af8ef32a57cc None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:07.320220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:07.320814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:07.321100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:07.321381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] PATH_INFO: `/v3/projects/e01fc25b705d4b08b61a87ec95e7c7b3` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:07.330092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01fc25b705d4b08b61a87ec95e7c7b3)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:07.333096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:07.531454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01fc25b705d4b08b61a87ec95e7c7b3', 'request_id': 'req-3d2f4d5d-427c-437b-9ecf-49f140a1f043'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:07.538994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01fc25b705d4b08b61a87ec95e7c7b3', 'request_id': 'req-3d2f4d5d-427c-437b-9ecf-49f140a1f043'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:07.667695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01fc25b705d4b08b61a87ec95e7c7b3', 'request_id': 'req-3d2f4d5d-427c-437b-9ecf-49f140a1f043'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:07.681741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] The token cache is being invalidate because project e01fc25b705d4b08b61a87ec95e7c7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:07.689587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:07.689587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:07.689587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 971/1938] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:07 2022] DELETE /identity/v3/projects/e01fc25b705d4b08b61a87ec95e7c7b3 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:07.693724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:07.704170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08304853-a9ba-4b40-970b-182206f44800 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:07.777673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:07.777673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:07.777673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:07.778018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] PATH_INFO: `/v3/users/b0cbc2c7b2be409ebe096b0c046e41b7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:07.780015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cbc2c7b2be409ebe096b0c046e41b7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:07.781798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:07.866965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:08.140564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cbc2c7b2be409ebe096b0c046e41b7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:08.152879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cbc2c7b2be409ebe096b0c046e41b7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:08.176613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cbc2c7b2be409ebe096b0c046e41b7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:08.258996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cbc2c7b2be409ebe096b0c046e41b7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:08.270390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cbc2c7b2be409ebe096b0c046e41b7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:08.288763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 968/1939] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:07 2022] DELETE /identity/v3/users/b0cbc2c7b2be409ebe096b0c046e41b7 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:08.989766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:08.992062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:09.011807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f4d5d-427c-437b-9ecf-49f140a1f043 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:09.094657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:09.095320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:09.095642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:09.095916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] PATH_INFO: `/v3/projects/0e7bfd3df2cb43ffb3f15cce594c59cc` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:09.098010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7bfd3df2cb43ffb3f15cce594c59cc)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:09.100409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:09.105427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:09.107061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:09.121522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e124-1345-46bb-9e9a-da897c12a387 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:09.129346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:09.130066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:09.130383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:09.130673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] PATH_INFO: `/v3/users/2b8cd9f063f344349332b216165628bf` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:09.131944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8cd9f063f344349332b216165628bf)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:09.133956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:09.301397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7bfd3df2cb43ffb3f15cce594c59cc', 'request_id': 'req-a80b919c-1d2f-471b-b136-8468d4dfed0c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.309202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7bfd3df2cb43ffb3f15cce594c59cc', 'request_id': 'req-a80b919c-1d2f-471b-b136-8468d4dfed0c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.438568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7bfd3df2cb43ffb3f15cce594c59cc', 'request_id': 'req-a80b919c-1d2f-471b-b136-8468d4dfed0c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.457298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] The token cache is being invalidate because project 0e7bfd3df2cb43ffb3f15cce594c59cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:09.457561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.458546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.458963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8cd9f063f344349332b216165628bf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.460192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8cd9f063f344349332b216165628bf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.462377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 972/1940] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:08 2022] DELETE /identity/v3/projects/0e7bfd3df2cb43ffb3f15cce594c59cc => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:09.467513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:09.470027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:09.478449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8cd9f063f344349332b216165628bf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.570021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b919c-1d2f-471b-b136-8468d4dfed0c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:09.575061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:09.575721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:09.576012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:09.576284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] PATH_INFO: `/v3/users/aab48e6052734bb4832f34846772a968` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:09.577736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab48e6052734bb4832f34846772a968)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:09.581292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:09.638161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8cd9f063f344349332b216165628bf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.641493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:09.653466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8cd9f063f344349332b216165628bf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:09.660357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 969/1941] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:09 2022] DELETE /identity/v3/users/2b8cd9f063f344349332b216165628bf => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:09.661213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:09.662613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:09.735287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe2c1-8b3e-4981-82d3-b1f50691b745 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:09.740559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:09.741215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:09.741496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:09.741772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] PATH_INFO: `/v3/users/8c64624b07c047dd972efaf953f66b3e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:09.742952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c64624b07c047dd972efaf953f66b3e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:09.744979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:09.872167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab48e6052734bb4832f34846772a968'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.016464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab48e6052734bb4832f34846772a968'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.043204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab48e6052734bb4832f34846772a968'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.043204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab48e6052734bb4832f34846772a968'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.060882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e0a94140-7a2d-4a9d-b9dd-cdac022bdba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab48e6052734bb4832f34846772a968'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.069723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 973/1942] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:09 2022] DELETE /identity/v3/users/aab48e6052734bb4832f34846772a968 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:10.074291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:10.074571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:10.074849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:10.100482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:10.101781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:10.175450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c64624b07c047dd972efaf953f66b3e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.177304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c64624b07c047dd972efaf953f66b3e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.181110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 974/1943] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:10.203921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:10.206738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:10.211864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c64624b07c047dd972efaf953f66b3e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.287842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fb9c-b8d0-4c62-9b8c-2f93a2496e49 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:10.295581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45fbbb-91cf-48fb-84bc-5ab656892b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:10.298610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45fbbb-91cf-48fb-84bc-5ab656892b91 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:10.299543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45fbbb-91cf-48fb-84bc-5ab656892b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:10.300275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45fbbb-91cf-48fb-84bc-5ab656892b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:10.312446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45fbbb-91cf-48fb-84bc-5ab656892b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:10.314785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45fbbb-91cf-48fb-84bc-5ab656892b91 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:10.331110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c64624b07c047dd972efaf953f66b3e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.342999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 975/1944] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:10.354963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c64624b07c047dd972efaf953f66b3e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:10.381575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 970/1945] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:09 2022] DELETE /identity/v3/users/8c64624b07c047dd972efaf953f66b3e => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:10.384290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:10.384832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:10.385383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:10.426199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:10.428313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:10.561225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 976/1946] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:10.590811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:10.593829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:10.758912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14d972-857b-4d4c-b0d0-e2dff7cfd0ad admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:10.765002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:10.766335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:10.766897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:10.768992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:10.816612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:10.827401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:10.830702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:10.866968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 971/1947] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:11.245322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:11.249684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:11.303418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320f639-5f6b-4e33-82c8-b381552c97fd None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:11.320103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:11.320918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:11.321363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:11.321751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] PATH_INFO: `/v3/projects/99082a3be3f64d2c938e04330fa7f7c0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:11.327126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99082a3be3f64d2c938e04330fa7f7c0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:11.333808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:11.403362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:11.405691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:11.425858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8a463-8e6b-46e9-98dc-d3614ff110d4 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:11.431830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:11.433317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:11.434991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:11.435494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] PATH_INFO: `/v3/projects/1d91ac516fff4c7f9933249775113c93` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:11.438140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d91ac516fff4c7f9933249775113c93)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:11.442006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:11.634325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99082a3be3f64d2c938e04330fa7f7c0', 'request_id': 'req-07658295-76b8-4842-bc59-ecc2b0dab788'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.649564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99082a3be3f64d2c938e04330fa7f7c0', 'request_id': 'req-07658295-76b8-4842-bc59-ecc2b0dab788'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.679006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d91ac516fff4c7f9933249775113c93', 'request_id': 'req-911593eb-87bc-4402-97e4-8fb2088d59d0'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.693451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d91ac516fff4c7f9933249775113c93', 'request_id': 'req-911593eb-87bc-4402-97e4-8fb2088d59d0'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.706968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d91ac516fff4c7f9933249775113c93', 'request_id': 'req-911593eb-87bc-4402-97e4-8fb2088d59d0'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.758721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99082a3be3f64d2c938e04330fa7f7c0', 'request_id': 'req-07658295-76b8-4842-bc59-ecc2b0dab788'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.772533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] The token cache is being invalidate because project 99082a3be3f64d2c938e04330fa7f7c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:11.772825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.773556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-07658295-76b8-4842-bc59-ecc2b0dab788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.775189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 977/1948] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:11 2022] DELETE /identity/v3/projects/99082a3be3f64d2c938e04330fa7f7c0 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:11.780162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:11.780446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:11.780725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:11.809436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:11.809436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:11.927802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 978/1949] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:11.927802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:11.927802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:11.937743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c09b7d-152a-4dd0-bea6-819a48c3f821 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:11.938823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] The token cache is being invalidate because project 1d91ac516fff4c7f9933249775113c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:11.940158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.944034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:11.944893 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:11.945700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:11.946145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] PATH_INFO: `/v3/projects/c0024c9db04e416e8f473af7b1ba0e83` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:11.947668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0024c9db04e416e8f473af7b1ba0e83)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:11.949533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-911593eb-87bc-4402-97e4-8fb2088d59d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:11.949925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:11.958796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 972/1950] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:11 2022] DELETE /identity/v3/projects/1d91ac516fff4c7f9933249775113c93 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:11.961523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:11.962455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:11.962979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:11.996006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:11.997642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:12.008705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 973/1951] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:12.009943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:12.011399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:12.028327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018a2d-d8a0-40a4-82a9-b998191857ab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:12.043509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:12.043509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:12.043509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:12.043509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] PATH_INFO: `/v3/users/0bfadcf3637a4e698e271b056212d46b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:12.044180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfadcf3637a4e698e271b056212d46b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:12.045261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:12.199627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0024c9db04e416e8f473af7b1ba0e83', 'request_id': 'req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.210095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0024c9db04e416e8f473af7b1ba0e83', 'request_id': 'req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.376847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0024c9db04e416e8f473af7b1ba0e83', 'request_id': 'req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.403924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] The token cache is being invalidate because project c0024c9db04e416e8f473af7b1ba0e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:12.403924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.403924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.403924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 979/1952] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:11 2022] DELETE /identity/v3/projects/c0024c9db04e416e8f473af7b1ba0e83 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:12.403924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:12.406766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:12.512348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfadcf3637a4e698e271b056212d46b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.519630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfadcf3637a4e698e271b056212d46b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.550368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfadcf3637a4e698e271b056212d46b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.554217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeac6b9-62c3-4bb5-af59-aa1e8a90873c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:12.563247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:12.564460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:12.564951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:12.565464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:12.566847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:12.567415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:12.571111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:12.599814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 980/1953] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:07:12.601770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:12.609284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:12.635990 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c64aa8-b50a-469d-a358-4a6bb9a07a54 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:12.643044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:12.643886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:12.645861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:12.646288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] PATH_INFO: `/v3/users/e84c60a210a6401eae47850954226cf1` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:12.648305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84c60a210a6401eae47850954226cf1)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:12.650649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:12.662584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfadcf3637a4e698e271b056212d46b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.683041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfadcf3637a4e698e271b056212d46b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:12.699667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 974/1954] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:12 2022] DELETE /identity/v3/users/0bfadcf3637a4e698e271b056212d46b => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:12.699667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:12.699667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:12.867493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268022b-67d2-471f-b99f-817b7392f860 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:12.895035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:12.896436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:12.896436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:12.896436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:12.897113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:12.898979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:12.982101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84c60a210a6401eae47850954226cf1'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:13.080500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 975/1955] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:12 2022] POST /identity/v3/projects => generated 369 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:13.086199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:13.114820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84c60a210a6401eae47850954226cf1'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:13.143970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84c60a210a6401eae47850954226cf1'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:13.148665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84c60a210a6401eae47850954226cf1'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:13.163530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84c60a210a6401eae47850954226cf1'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:13.175662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 981/1956] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:12 2022] DELETE /identity/v3/users/e84c60a210a6401eae47850954226cf1 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:13.225172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bc8fc-5c05-433c-8ec7-4272dc03f5ac admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:13.365262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:13.366439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:13.367793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:13.367793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:13.371414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:13.375851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:13.707239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 976/1957] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:13 2022] POST /identity/v3/users => generated 285 bytes in 622 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:13.711371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:13.713139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:13.731596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ace51-314b-4d03-83cc-8a418b5cb02e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:13.739105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:13.740003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:13.740400 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:13.740834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:13.741613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:13.758308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:13.761094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:13.764688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 982/1958] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:13.778535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:13.888162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99ec12-0691-4f3d-934f-34b7b7f656e5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:13.888162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:13.888162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:13.888162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:13.889697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:13.889697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:13.889697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:13.889697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 977/1959] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:13.895315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:13.910115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e781e-11d7-4882-a40b-26c0383bfa66 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:13.916476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:13.920193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:13.920193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:13.920193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] PATH_INFO: `/v3/projects/e0f123b8ea4747f7aae0ddc146019cd6/users/d6aa518cc8004be6b79c82623574b056/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:13.953034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f123b8ea4747f7aae0ddc146019cd6, user_id=d6aa518cc8004be6b79c82623574b056, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:13.959143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:14.182610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:14.184959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:14.205455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b037774-d82f-4acb-8a2e-e55d6e7c5d8e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:14.211318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:14.212200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:14.212590 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:14.213122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] PATH_INFO: `/v3/projects/2447d6122b314ee293bf4cbbf4381a1b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:14.214968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2447d6122b314ee293bf4cbbf4381a1b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:14.217494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:14.353250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 983/1960] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:13 2022] PUT /identity/v3/projects/e0f123b8ea4747f7aae0ddc146019cd6/users/d6aa518cc8004be6b79c82623574b056/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:14.353250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:14.373561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:14.524165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1d41-6ac7-4634-8f39-ab49a00e1f9e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:14.542697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2447d6122b314ee293bf4cbbf4381a1b', 'request_id': 'req-19cb525e-28c8-4f8e-9420-87260a37faac'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:14.555505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2447d6122b314ee293bf4cbbf4381a1b', 'request_id': 'req-19cb525e-28c8-4f8e-9420-87260a37faac'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:14.575623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2447d6122b314ee293bf4cbbf4381a1b', 'request_id': 'req-19cb525e-28c8-4f8e-9420-87260a37faac'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:14.645124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:14.645921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:14.646853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:14.646853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] PATH_INFO: `/v3/projects/4c311fec58c541df9cc1c86e1c1321e8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:14.648112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c311fec58c541df9cc1c86e1c1321e8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:14.650618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:14.761308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] The token cache is being invalidate because project 2447d6122b314ee293bf4cbbf4381a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:14.761308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:14.761308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19cb525e-28c8-4f8e-9420-87260a37faac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:14.763207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 978/1961] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:14 2022] DELETE /identity/v3/projects/2447d6122b314ee293bf4cbbf4381a1b => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:14.765474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491acbf-259b-44f2-b748-a19e9df3e43d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:14.766097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491acbf-259b-44f2-b748-a19e9df3e43d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:14.766576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491acbf-259b-44f2-b748-a19e9df3e43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:14.803751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1491acbf-259b-44f2-b748-a19e9df3e43d None None] MFA Rules not processed for user `d6aa518cc8004be6b79c82623574b056`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:14.805963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1491acbf-259b-44f2-b748-a19e9df3e43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:14.930424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 979/1962] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:14.932270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:14.933038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:14.933465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:14.942896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c311fec58c541df9cc1c86e1c1321e8', 'request_id': 'req-b1298d80-2126-45f8-bb52-d8ecf46aa37f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:14.960712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:14.962004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:14.969849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c311fec58c541df9cc1c86e1c1321e8', 'request_id': 'req-b1298d80-2126-45f8-bb52-d8ecf46aa37f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:15.010090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:15.091112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c311fec58c541df9cc1c86e1c1321e8', 'request_id': 'req-b1298d80-2126-45f8-bb52-d8ecf46aa37f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:15.107560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 980/1963] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:15.107560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.107560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.144750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] The token cache is being invalidate because project 4c311fec58c541df9cc1c86e1c1321e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:15.145188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:15.145839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b1298d80-2126-45f8-bb52-d8ecf46aa37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:15.148176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 984/1964] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:14 2022] DELETE /identity/v3/projects/4c311fec58c541df9cc1c86e1c1321e8 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:15.153770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bc6a2-96cc-43fa-8835-2f0f7723892c None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.154340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bc6a2-96cc-43fa-8835-2f0f7723892c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.154878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bc6a2-96cc-43fa-8835-2f0f7723892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.208238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-114bc6a2-96cc-43fa-8835-2f0f7723892c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:15.211363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-114bc6a2-96cc-43fa-8835-2f0f7723892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.223314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 985/1965] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:15.225623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.226137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.226556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.271746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:15.273552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.287782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 986/1966] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:15.288968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.290182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1771a33-dfc6-474d-a406-c6781d90dbe3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.297601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.300103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.300541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.301003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.304193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.308109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feab04e-ccab-44e8-b2da-8bb18c44fef1 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.315535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.317154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.317607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.318779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.320370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:15.320878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.323062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bedbac-3c48-43f4-a300-84cd31414fe6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.349016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 987/1967] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:07:15.350713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.351226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.352647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.385508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:15.387319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.398608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 988/1968] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:15.399821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.411896 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b54f0-90e0-47b5-a84a-7c07e0dc7c13 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.417376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.421702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.421702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.421702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.421702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.422731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.422958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.425635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.439177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 981/1969] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:15.440510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.463309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5772da-fb0d-48ca-833c-e5efa13c2eac service glance] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.479097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.480544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.481277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.486269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.495957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:15.495957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.495957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.520018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 982/1970] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:07:15.520018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.543061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67e400-864d-48aa-b9d8-f2f7c2dc812a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.548548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.549492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.550101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.550526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.551338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.554048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.660695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 989/1971] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:15.665356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.761929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 983/1972] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/projects => generated 377 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:15.773353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.803590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386fb87-b2f1-4cef-8ca3-1885fa7d5246 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.899131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61066ad-d59b-44cb-9ffb-73a8111a8442 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.922757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.923904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.924406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.924911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.926125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.927344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.928270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.928620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.929002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.929946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.930623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.932870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.103301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 990/1974] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/users => generated 284 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:16.105546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.106297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 984/1974] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:16.110156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.132211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aedc3a-3cbb-4eee-b615-9d66b217f9a3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:16.137510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edfe57-9290-47dd-a67f-7be34f635423 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:16.143337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:16.143890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.144031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.144548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.144947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:16.146721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.156193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:16.157371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.157861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.158246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.159452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:16.161332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.176835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 985/1975] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:16.179747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.197381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 991/1976] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:16.198673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.212209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ef59-8324-4b15-9434-9a47f151b0d3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:16.219455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:16.220270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.220741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.221067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa63b50-85f6-4dab-a59e-ab9c0249a579 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:16.221244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.222070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:16.223778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.238744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:16.239424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.239709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.239989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.240804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:16.244895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.246044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 986/1977] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:16.247367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.261705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88579c2b-ad5c-4671-82dc-1a53da95b3ad admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:16.266370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 992/1978] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:16.267384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.269022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6251-b0b5-4741-afbc-4027def530d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:16.270240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6251-b0b5-4741-afbc-4027def530d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.274844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6251-b0b5-4741-afbc-4027def530d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.274844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6251-b0b5-4741-afbc-4027def530d7 admin admin] PATH_INFO: `/v3/projects/8954bd94261f469a8e7851b3a3125771/users/1b8f03e4390a4725a97ea299575a175a/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.288016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710ed3-7402-4c59-96e8-fcdd6827ff1a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:16.297385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6251-b0b5-4741-afbc-4027def530d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8954bd94261f469a8e7851b3a3125771, user_id=1b8f03e4390a4725a97ea299575a175a, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:16.299098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eb83b-fac2-4c38-bdae-2d0249e41b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:16.300017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eb83b-fac2-4c38-bdae-2d0249e41b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.300324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eb83b-fac2-4c38-bdae-2d0249e41b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.300596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eb83b-fac2-4c38-bdae-2d0249e41b80 admin admin] PATH_INFO: `/v3/projects/6808847ac5e54d87b314e298acd9fe0d/users/90a2c5810dcb4a16aa05702da0d28a9e/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.302358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6251-b0b5-4741-afbc-4027def530d7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.333969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eb83b-fac2-4c38-bdae-2d0249e41b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6808847ac5e54d87b314e298acd9fe0d, user_id=90a2c5810dcb4a16aa05702da0d28a9e, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:16.333969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eb83b-fac2-4c38-bdae-2d0249e41b80 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:16.511466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 987/1979] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:16 2022] PUT /identity/v3/projects/8954bd94261f469a8e7851b3a3125771/users/1b8f03e4390a4725a97ea299575a175a/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:16.533438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.533969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.534401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.583564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] MFA Rules not processed for user `1b8f03e4390a4725a97ea299575a175a`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:16.585061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:16.622588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 993/1980] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:16 2022] PUT /identity/v3/projects/6808847ac5e54d87b314e298acd9fe0d/users/90a2c5810dcb4a16aa05702da0d28a9e/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:16.641451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:16.641730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:16.642382 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:16.708119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] MFA Rules not processed for user `90a2c5810dcb4a16aa05702da0d28a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:16.710661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:16.759531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 988/1981] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:16.765595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.876317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 994/1982] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:16.882534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:16.936219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994f27d-f9dd-4838-96f7-d7349e92e853 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.008386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7cffa-fae5-4801-a92d-629e8fcb660b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.043476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10826ce-967d-46b8-bf46-83eb4ad27913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.043476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10826ce-967d-46b8-bf46-83eb4ad27913 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.043476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10826ce-967d-46b8-bf46-83eb4ad27913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.043476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10826ce-967d-46b8-bf46-83eb4ad27913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.043863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10826ce-967d-46b8-bf46-83eb4ad27913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.054711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10826ce-967d-46b8-bf46-83eb4ad27913 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.056488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145533b9-ed48-47fd-8fa5-5292eb1b0a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.057637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145533b9-ed48-47fd-8fa5-5292eb1b0a2b admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.058234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145533b9-ed48-47fd-8fa5-5292eb1b0a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.058740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145533b9-ed48-47fd-8fa5-5292eb1b0a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.060124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145533b9-ed48-47fd-8fa5-5292eb1b0a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.064904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145533b9-ed48-47fd-8fa5-5292eb1b0a2b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.233320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 989/1983] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:16 2022] POST /identity/v3/projects => generated 367 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:17.235300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e59f-2f68-475c-b386-97d16cc08f66 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.235860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e59f-2f68-475c-b386-97d16cc08f66 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.236085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e59f-2f68-475c-b386-97d16cc08f66 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.238470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 990/1984] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:07:17.239785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32e59f-2f68-475c-b386-97d16cc08f66 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.282904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:17.287789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 995/1985] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:16 2022] POST /identity/v3/projects => generated 375 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:17.291118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.291455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.292016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.317657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:17.318996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:17.340087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32e59f-2f68-475c-b386-97d16cc08f66 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.418884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.420102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.420570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.420922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.421763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.422204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 996/1986] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:17.423265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.427856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.439212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4dcf-a53a-4176-9e29-73cf2de9a830 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.446391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2a67-d3dd-43eb-aa29-5486890bf8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.447138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2a67-d3dd-43eb-aa29-5486890bf8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.447597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2a67-d3dd-43eb-aa29-5486890bf8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.447989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2a67-d3dd-43eb-aa29-5486890bf8ac admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.448858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a2a67-d3dd-43eb-aa29-5486890bf8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.451126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a2a67-d3dd-43eb-aa29-5486890bf8ac admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.559737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 991/1987] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:17 2022] POST /identity/v3/users => generated 290 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:17.560777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.563377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:17.579122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc53ad-d656-45dd-81aa-689acea54644 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.585803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.586683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.587170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.587570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] PATH_INFO: `/v3/users/c5b991f5024e4b5dbe8510b7381b1dc8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.589056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b991f5024e4b5dbe8510b7381b1dc8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.591067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.633882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 997/1988] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:17 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:17.634906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b39f-4db9-4386-ad1b-f48728b19cd4 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.635178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b39f-4db9-4386-ad1b-f48728b19cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.635781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b39f-4db9-4386-ad1b-f48728b19cd4 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.638562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 998/1989] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 1 14:07:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:07:17.640234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65b39f-4db9-4386-ad1b-f48728b19cd4 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.657242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65b39f-4db9-4386-ad1b-f48728b19cd4 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.663084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.663826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.664038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.664318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.665022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.666858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.678922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 999/1990] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:17.679847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.691524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125325ee-79f1-462c-8d84-60519b8c5cfa admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.699532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d450-a9c1-459e-8c91-e35d1b41c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.704694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d450-a9c1-459e-8c91-e35d1b41c090 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.704694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d450-a9c1-459e-8c91-e35d1b41c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.704694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d450-a9c1-459e-8c91-e35d1b41c090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.704694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90d450-a9c1-459e-8c91-e35d1b41c090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.707785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90d450-a9c1-459e-8c91-e35d1b41c090 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.723821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1000/1991] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:17.724091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.724482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.724675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.747932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] MFA Rules not processed for user `51e7d81890cc4ee5a357765f2a2bbbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:17.748626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:17.760993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1001/1992] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 1 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:17.764703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.784092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517b599-77ff-4a69-a1d4-f223a00a4c1e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.789759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.790606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.791068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.791353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] PATH_INFO: `/v3/projects/09962d174f96449f9daf5aa2b879988e/users/27b602edc29b470e97a15c25214ccce9/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.822141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09962d174f96449f9daf5aa2b879988e, user_id=27b602edc29b470e97a15c25214ccce9, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:17.825046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:17.883644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b991f5024e4b5dbe8510b7381b1dc8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:17.888215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b991f5024e4b5dbe8510b7381b1dc8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:17.910570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b991f5024e4b5dbe8510b7381b1dc8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:17.957022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1002/1993] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:17 2022] PUT /identity/v3/projects/09962d174f96449f9daf5aa2b879988e/users/27b602edc29b470e97a15c25214ccce9/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:17.958110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.022618 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b991f5024e4b5dbe8510b7381b1dc8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:18.030401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f925332-4e71-4e14-820b-73993c6df7d0 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.034463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b991f5024e4b5dbe8510b7381b1dc8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:18.036658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.037403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.037702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.038012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] PATH_INFO: `/v3/projects/322bec89bbe04a4d9de85fd35d7b1338/users/b2614bb3b7f441c498eca17977814497/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.044358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 992/1994] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:17 2022] DELETE /identity/v3/users/c5b991f5024e4b5dbe8510b7381b1dc8 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:18.052980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.055862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.059207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322bec89bbe04a4d9de85fd35d7b1338, user_id=b2614bb3b7f441c498eca17977814497, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.061532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.192556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2125d-b072-47ce-a773-314da7be058b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.200328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1003/1995] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:17 2022] PUT /identity/v3/projects/322bec89bbe04a4d9de85fd35d7b1338/users/b2614bb3b7f441c498eca17977814497/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:18.200328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.200328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.324039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:18.331427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.332395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.332930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.333354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.336027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.348448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8ea3d-1076-422c-83ab-28e283dda3a9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.357386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.374892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.381315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.381315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.382050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.382154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.382154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.385129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.407267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1004/1996] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:18.408649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 993/1997] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:18.408833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.411437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.422467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.447551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98584d-4f61-42e6-b613-3b849633b229 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.447809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16642-6308-4d1b-adfa-71748b727c12 service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.451462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.464982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.464982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.464982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.464982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.465337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.465457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.465552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.465552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.465552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] PATH_INFO: `/v3/projects/09962d174f96449f9daf5aa2b879988e/users/27b602edc29b470e97a15c25214ccce9/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.465552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] RBAC: Authorizing `identity:create_grant(project_id=09962d174f96449f9daf5aa2b879988e, user_id=27b602edc29b470e97a15c25214ccce9, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.469093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.477958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1005/1998] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:18.483170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 994/1999] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:18 2022] PUT /identity/v3/projects/09962d174f96449f9daf5aa2b879988e/users/27b602edc29b470e97a15c25214ccce9/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:18.493559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.493752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.499191 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.508903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c84033-6b76-481f-9581-7800c5fe2b21 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.617372 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdaaff-946c-42ac-a922-a9485b634375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.618364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdaaff-946c-42ac-a922-a9485b634375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.618698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdaaff-946c-42ac-a922-a9485b634375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.619018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdaaff-946c-42ac-a922-a9485b634375 admin admin] PATH_INFO: `/v3/projects/322bec89bbe04a4d9de85fd35d7b1338/users/b2614bb3b7f441c498eca17977814497/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.621295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdaaff-946c-42ac-a922-a9485b634375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322bec89bbe04a4d9de85fd35d7b1338, user_id=b2614bb3b7f441c498eca17977814497, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.625195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdaaff-946c-42ac-a922-a9485b634375 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.632034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269718-28c2-4e4d-a3a7-42dbcf2d52de admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:18.638874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1006/2000] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:18 2022] PUT /identity/v3/projects/322bec89bbe04a4d9de85fd35d7b1338/users/b2614bb3b7f441c498eca17977814497/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:18.640457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716504ef-61ec-4af2-bb20-ef4cb5fdc2c9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.640764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716504ef-61ec-4af2-bb20-ef4cb5fdc2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.640999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716504ef-61ec-4af2-bb20-ef4cb5fdc2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.682022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-716504ef-61ec-4af2-bb20-ef4cb5fdc2c9 None None] MFA Rules not processed for user `27b602edc29b470e97a15c25214ccce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:18.683496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-716504ef-61ec-4af2-bb20-ef4cb5fdc2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.751273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:18.752363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.752953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.753504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] PATH_INFO: `/v3/projects/f871b0e1d3154b6d8c297b12ae738be0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.755529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f871b0e1d3154b6d8c297b12ae738be0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.759524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.794592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1007/2001] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:18.796029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:18.796319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:18.796599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:18.840754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] MFA Rules not processed for user `b2614bb3b7f441c498eca17977814497`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:18.842767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.935101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1008/2002] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:18.943474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:18.947730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:19.022220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f871b0e1d3154b6d8c297b12ae738be0', 'request_id': 'req-d21bff56-8389-46ed-82ce-29e543d228e8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:19.042823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f871b0e1d3154b6d8c297b12ae738be0', 'request_id': 'req-d21bff56-8389-46ed-82ce-29e543d228e8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:19.056747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f871b0e1d3154b6d8c297b12ae738be0', 'request_id': 'req-d21bff56-8389-46ed-82ce-29e543d228e8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:19.087426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c9f82-6229-4867-b3f3-62b1cf101ec8 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:19.170174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] The token cache is being invalidate because project f871b0e1d3154b6d8c297b12ae738be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:19.170687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:19.172028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:19.175248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 995/2003] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:18 2022] DELETE /identity/v3/projects/f871b0e1d3154b6d8c297b12ae738be0 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:19.176785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:19.181133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:19.190301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:19.193123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.193123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.193123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.194716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:19.198963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bff56-8389-46ed-82ce-29e543d228e8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:19.203161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:19.203821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.204131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.204544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.208373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:19.284885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:19.286721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f478c8-5731-488f-b2a9-b1f7d4618599 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:19.303582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 996/2004] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:19.306199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.306390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:19.306968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.307407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.308094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36b92c-2c09-4b30-ad8d-1f30e74c9fa2 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:19.324468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1009/2005] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:19.348515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:19.351389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:19.438004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 997/2006] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:19.447436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.447770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.449979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.471668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:19.473254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:19.486862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1010/2007] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:19.498335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:19.521133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77248e-d294-4116-ba66-d1f327ac23d1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:19.528363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:19.529403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.529792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.530218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.531146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:19.531553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:19.534620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:19.550285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 998/2008] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:07:19.555098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:19.570103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce353e-3f25-4d6f-9ad2-db1a821ec8d5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:19.599210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:19.599210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.599210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.599210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.599772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:19.599772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:19.785261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1011/2009] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:19 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:19.786818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:19.806699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957cba-dbd4-43bf-97a4-0cdcae13a90b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:19.916193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:19.917040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:19.917447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:19.917937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:19.919082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:19.922027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:20.040294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 999/2010] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:19 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:20.046399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:20.063039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ea5b9-b820-45a9-88ca-81fc28605842 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:20.070041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:20.075188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:20.075188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:20.075188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:20.075188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:20.075188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:20.085720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1012/2011] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:20.090831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:20.105764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d0a8a-d4bd-4145-9be2-c67ec2bccadd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:20.109546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:20.113107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:20.113744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:20.114642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:20.114938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:20.115216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:20.115907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:20.117708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:20.138010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1000/2012] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:07:20.145709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:20.164857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c55dc-9653-439d-bea1-c47cd43df83f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:20.186317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b2dca-1adf-481f-b0ba-59ce47539978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:20.187429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b2dca-1adf-481f-b0ba-59ce47539978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:20.187770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b2dca-1adf-481f-b0ba-59ce47539978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:20.188335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b2dca-1adf-481f-b0ba-59ce47539978 admin admin] PATH_INFO: `/v3/projects/90a84500a899479aabfa3aeb72b4c4ae/users/7ae66df61a764a42a1b1b15f54d5c34b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:20.217784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b2dca-1adf-481f-b0ba-59ce47539978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a84500a899479aabfa3aeb72b4c4ae, user_id=7ae66df61a764a42a1b1b15f54d5c34b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:20.222069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1a787-9a70-4b0b-89bd-9e4908f42ee3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:20.223966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b2dca-1adf-481f-b0ba-59ce47539978 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:20.228671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:20.229783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:20.230187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:20.230642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:20.236305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:20.383563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1001/2013] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:20 2022] PUT /identity/v3/projects/90a84500a899479aabfa3aeb72b4c4ae/users/7ae66df61a764a42a1b1b15f54d5c34b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:20.398074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:20.398569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:20.399040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:20.402860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:20.407887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:20.445832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1013/2014] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:20.466826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:20.471580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:20.587348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1002/2015] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:20.599927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:20.601928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:20.693225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715a88b-132f-405f-8d06-9537ee4fa608 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:20.772185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:20.780278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981c925-54c1-441f-809f-40d908425747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:20.782855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c925-54c1-441f-809f-40d908425747 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:20.782855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c925-54c1-441f-809f-40d908425747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:20.782855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c925-54c1-441f-809f-40d908425747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:20.794518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981c925-54c1-441f-809f-40d908425747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:20.796357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981c925-54c1-441f-809f-40d908425747 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:20.813995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1014/2016] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:21.029704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:21.441049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:21.443183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:21.462506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80aa70-b45e-41d1-ac1b-f59bf8363afb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:21.472480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:21.473748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:21.474275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:21.474797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] PATH_INFO: `/v3/users/c189a6afc7564155944552831350b979` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:21.477150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c189a6afc7564155944552831350b979)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:21.481542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:21.856426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c189a6afc7564155944552831350b979'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:21.857295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c189a6afc7564155944552831350b979'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:21.875671 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c189a6afc7564155944552831350b979'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:21.993111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c189a6afc7564155944552831350b979'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:22.006757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c189a6afc7564155944552831350b979'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:22.023428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1003/2017] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:21 2022] DELETE /identity/v3/users/c189a6afc7564155944552831350b979 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:22.030852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:22.031417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:22.031810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:22.064941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:22.066979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:22.171464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1015/2018] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:22.183454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:22.199056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:22.292226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e3ab-8000-4ace-918e-b7a85bcc6690 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:22.298902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec9f3e-a089-460f-b2b0-e16e6548a86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:22.300194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec9f3e-a089-460f-b2b0-e16e6548a86d service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:22.300599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec9f3e-a089-460f-b2b0-e16e6548a86d service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:22.300922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec9f3e-a089-460f-b2b0-e16e6548a86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:22.312212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec9f3e-a089-460f-b2b0-e16e6548a86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:22.313820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec9f3e-a089-460f-b2b0-e16e6548a86d service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:22.325932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1004/2019] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:22.716987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:22.853167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:23.312908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:23.312908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:23.362910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8812f-fdb6-48f4-9457-ea96ab5208cb None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:23.362910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:23.362910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:23.362910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:23.363442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] PATH_INFO: `/v3/projects/37239e3218ed49a69f17e3231ceabbcf` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:23.364458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=37239e3218ed49a69f17e3231ceabbcf)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:23.366775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:23.623216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37239e3218ed49a69f17e3231ceabbcf', 'request_id': 'req-88767792-0216-4a57-be4d-6702ca31f65a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:23.631929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37239e3218ed49a69f17e3231ceabbcf', 'request_id': 'req-88767792-0216-4a57-be4d-6702ca31f65a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:23.696145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:23.770352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37239e3218ed49a69f17e3231ceabbcf', 'request_id': 'req-88767792-0216-4a57-be4d-6702ca31f65a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:23.787673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] The token cache is being invalidate because project 37239e3218ed49a69f17e3231ceabbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:23.788349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:23.789477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-88767792-0216-4a57-be4d-6702ca31f65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:23.792169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1016/2020] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:23 2022] DELETE /identity/v3/projects/37239e3218ed49a69f17e3231ceabbcf => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:23.801333 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:23.806606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:23.807393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:23.839441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:23.843143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:23.935806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1005/2021] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:23.945583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:23.945982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:23.946248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:23.974793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:23.976215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:23.985224 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:23.989289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1017/2022] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:23.991138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:24.016062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a05d4-5f24-4606-9043-0c8c9a1744d8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:24.031056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:24.032484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:24.033011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:24.033709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:24.035088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:24.035601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:24.038830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:24.065153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1006/2023] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:07:24.073004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:24.092456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ca59-e184-4c2d-9de3-5d224871bf68 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:24.100417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:24.101376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:24.101606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:24.104493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:24.104493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:24.104493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:24.256687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1018/2024] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:24 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:24.258517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:24.273081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8577f-e930-4710-b58b-f5cf167ddb67 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:24.390944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:24.391587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:24.391872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:24.392153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:24.393608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:24.395819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:24.614110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1007/2025] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:24 2022] POST /identity/v3/users => generated 288 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:07:24.632069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:24.649815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea93174-ef8a-434b-9210-1f919d55cb1d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:24.655142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:24.655778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:24.656296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:24.656756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:24.658233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:24.660875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:24.684469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1019/2026] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:24.692089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:24.706358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e822462-e44d-4ea6-9956-90a38aae58a9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:24.712923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120b53e-858a-4c39-a9d8-ed7ef4d3cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:24.714199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b53e-858a-4c39-a9d8-ed7ef4d3cbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:24.714568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b53e-858a-4c39-a9d8-ed7ef4d3cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:24.714917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b53e-858a-4c39-a9d8-ed7ef4d3cbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:24.715668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120b53e-858a-4c39-a9d8-ed7ef4d3cbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:24.717525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120b53e-858a-4c39-a9d8-ed7ef4d3cbe3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:24.752833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1008/2027] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:24.765164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:24.801412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02453f9f-1516-4a91-b135-bb6a66d9a295 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:24.835120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:24.836140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:24.836430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:24.836707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] PATH_INFO: `/v3/projects/1c13f664f15145fdb9b3c4b9f8496d80/users/446af6bea953401ead7e8a9c9604d8d0/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:24.878449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c13f664f15145fdb9b3c4b9f8496d80, user_id=446af6bea953401ead7e8a9c9604d8d0, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:24.880664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:24.997095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1020/2028] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:24 2022] PUT /identity/v3/projects/1c13f664f15145fdb9b3c4b9f8496d80/users/446af6bea953401ead7e8a9c9604d8d0/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:25.015365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:25.015976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:25.015976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:25.092343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] MFA Rules not processed for user `446af6bea953401ead7e8a9c9604d8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:25.094724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:25.210783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1009/2029] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:25.228023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:25.231121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:25.315325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30bbc3-10c2-484f-8baa-ee9626711aef admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:25.320010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b55c6-ce5c-4cd4-b2f2-a231dea98fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:25.320621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b55c6-ce5c-4cd4-b2f2-a231dea98fce service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:25.320932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b55c6-ce5c-4cd4-b2f2-a231dea98fce service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:25.321193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b55c6-ce5c-4cd4-b2f2-a231dea98fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:25.333128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b55c6-ce5c-4cd4-b2f2-a231dea98fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:25.334708 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b55c6-ce5c-4cd4-b2f2-a231dea98fce service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:25.353663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1021/2030] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:25.854957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:26.319867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:26.683860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:26.711569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbf88-c576-4543-942d-978d4811f4ec None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:26.717508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:26.719462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:26.720427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:26.721237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:26.724064 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:26.867541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:26.884774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e52280-50b1-497a-8fcc-c8289e208d6e service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:26.914616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1010/2031] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:27.711319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:27.711856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:27.711979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:27.746878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] MFA Rules not processed for user `446af6bea953401ead7e8a9c9604d8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:27.748215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:27.758060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1022/2032] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:28.379008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:29.016366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:29.756741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:29.757546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:29.766767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:29.799855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] MFA Rules not processed for user `7ae66df61a764a42a1b1b15f54d5c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:29.801574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:29.931174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1011/2033] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:29.949491 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:29.954833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:30.092337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:30.094299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:30.132414 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aac2d-64ac-4072-b9d2-e18848b2edfe None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:30.163985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:30.165235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:30.165791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:30.167474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] PATH_INFO: `/v3/users/1b8f03e4390a4725a97ea299575a175a` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:30.171321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8f03e4390a4725a97ea299575a175a)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:30.188279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:30.299675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043ef28-8285-44b6-88c7-1cba6adcf51f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:30.310964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262b246-ca17-4776-b1ab-cff7cd7115be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:30.318043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262b246-ca17-4776-b1ab-cff7cd7115be service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:30.318043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262b246-ca17-4776-b1ab-cff7cd7115be service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:30.318043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262b246-ca17-4776-b1ab-cff7cd7115be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:30.327673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262b246-ca17-4776-b1ab-cff7cd7115be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:30.331015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262b246-ca17-4776-b1ab-cff7cd7115be service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:30.345086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1012/2034] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:30.609039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8f03e4390a4725a97ea299575a175a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:30.717612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8f03e4390a4725a97ea299575a175a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:30.746589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8f03e4390a4725a97ea299575a175a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:30.747789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8f03e4390a4725a97ea299575a175a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:30.763505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8f03e4390a4725a97ea299575a175a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:30.778413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1023/2035] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:29 2022] DELETE /identity/v3/users/1b8f03e4390a4725a97ea299575a175a => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:30.780579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:30.780865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:30.781137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:30.804479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:30.806158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:30.910343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1013/2036] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:30.922766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:30.926190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:31.005061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b609a5-f8c2-4bf7-a739-e85ba632c554 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:31.010421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:31.011272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:31.011695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:31.012595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:31.028935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:31.030657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:31.051778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1024/2037] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:31.655691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:31.658348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:31.663950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:31.684018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060f9a3-9f24-434e-9b61-97c93b02d704 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:31.698973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:31.701143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:31.702346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:31.704131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] PATH_INFO: `/v3/projects/8954bd94261f469a8e7851b3a3125771` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:31.709072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8954bd94261f469a8e7851b3a3125771)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:31.714451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:31.943035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8954bd94261f469a8e7851b3a3125771', 'request_id': 'req-57208945-ceaf-457a-afaf-a749f2b593ef'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:31.950757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8954bd94261f469a8e7851b3a3125771', 'request_id': 'req-57208945-ceaf-457a-afaf-a749f2b593ef'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:31.960245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8954bd94261f469a8e7851b3a3125771', 'request_id': 'req-57208945-ceaf-457a-afaf-a749f2b593ef'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.126137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] The token cache is being invalidate because project 8954bd94261f469a8e7851b3a3125771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:32.126377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.127280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.129368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1014/2038] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:31 2022] DELETE /identity/v3/projects/8954bd94261f469a8e7851b3a3125771 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:32.134483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:32.150825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181faea0-7bf9-478b-9ee5-dc5eb407e7d0 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:32.242606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:32.243631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:32.244253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:32.244640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] PATH_INFO: `/v3/users/27b602edc29b470e97a15c25214ccce9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:32.246157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b602edc29b470e97a15c25214ccce9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:32.248468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:32.400819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:32.563218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b602edc29b470e97a15c25214ccce9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.686615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b602edc29b470e97a15c25214ccce9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.737231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b602edc29b470e97a15c25214ccce9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.738643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b602edc29b470e97a15c25214ccce9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.758503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ac9fbdd4-f360-45df-8fb3-bb076119658b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b602edc29b470e97a15c25214ccce9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:32.802179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1025/2039] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:32 2022] DELETE /identity/v3/users/27b602edc29b470e97a15c25214ccce9 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:33.704045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:33.706899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:33.725402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57208945-ceaf-457a-afaf-a749f2b593ef admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:33.883010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:33.891111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:33.891455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:33.891903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] PATH_INFO: `/v3/projects/09962d174f96449f9daf5aa2b879988e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:33.894151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09962d174f96449f9daf5aa2b879988e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:33.897119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:34.052508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09962d174f96449f9daf5aa2b879988e', 'request_id': 'req-5abbd46e-6c86-45fa-b237-c154cc6f299c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:34.062887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09962d174f96449f9daf5aa2b879988e', 'request_id': 'req-5abbd46e-6c86-45fa-b237-c154cc6f299c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:34.078058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09962d174f96449f9daf5aa2b879988e', 'request_id': 'req-5abbd46e-6c86-45fa-b237-c154cc6f299c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:34.224458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] The token cache is being invalidate because project 09962d174f96449f9daf5aa2b879988e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:07:34.224912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:34.227119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5abbd46e-6c86-45fa-b237-c154cc6f299c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:34.236793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:34.237575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:34.237575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:34.238006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1015/2040] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:07:33 2022] DELETE /identity/v3/projects/09962d174f96449f9daf5aa2b879988e => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:34.300648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:34.302594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:34.506313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1026/2041] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:34.514074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:34.514599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:34.515052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:34.554347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:34.556196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:34.570690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1016/2042] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:34.586615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:34.655659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b17ad-8dde-40fc-88bf-0a37acec33ec None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:34.675060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:34.679591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:34.681742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:34.682740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:34.683886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:07:34.684592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:34.688478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:34.728325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1027/2043] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:07:34.732948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:34.745114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ea884-0d80-41ff-a42c-96958b631737 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:34.751190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:34.751839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:34.752924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:34.753267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:34.754831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:34.756939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:34.968463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1017/2044] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:34 2022] POST /identity/v3/projects => generated 371 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:34.972490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:34.989727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a037a6-42ac-4427-9dc9-c2dbd2fa882c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:35.087347 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202379b3-5008-4984-b3f3-c8eb8b9da0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:35.087987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202379b3-5008-4984-b3f3-c8eb8b9da0dc admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:35.088269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202379b3-5008-4984-b3f3-c8eb8b9da0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:35.088544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202379b3-5008-4984-b3f3-c8eb8b9da0dc admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:35.089203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202379b3-5008-4984-b3f3-c8eb8b9da0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:35.091659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202379b3-5008-4984-b3f3-c8eb8b9da0dc admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:35.232030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1028/2045] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:34 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:35.236906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:35.260824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a0b5d-3651-472d-b6bd-210fcf055c26 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:35.275062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:35.276167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:35.276647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:35.277234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:35.278538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:35.281729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:35.296718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e88ffe-e8be-4a34-ad35-2fdcc00707f8 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:35.311312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e88ffe-e8be-4a34-ad35-2fdcc00707f8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:35.312929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e88ffe-e8be-4a34-ad35-2fdcc00707f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:35.318199 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1018/2046] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:35.322334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:35.352542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b004ea-99ec-4681-9420-e75e02402874 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:35.358645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:35.359393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:35.359782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:35.360168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:35.361155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:35.363745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:35.377126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-71e88ffe-e8be-4a34-ad35-2fdcc00707f8 None None] MFA Rules not processed for user `7ae66df61a764a42a1b1b15f54d5c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:35.379787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1019/2047] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:35.380368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-71e88ffe-e8be-4a34-ad35-2fdcc00707f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:35.385722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:35.397667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4adc-25e3-4d59-99bf-b9b64604d3c5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:35.405819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:35.406568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:35.406854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:35.407137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] PATH_INFO: `/v3/projects/404bac8465994797b72d704bba719f31/users/f77bc33a9f3e4fad9b533794c82390b5/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:35.446177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=404bac8465994797b72d704bba719f31, user_id=f77bc33a9f3e4fad9b533794c82390b5, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:35.450033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:35.494123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1029/2048] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:35.667747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1020/2049] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:35 2022] PUT /identity/v3/projects/404bac8465994797b72d704bba719f31/users/f77bc33a9f3e4fad9b533794c82390b5/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:07:35.677165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3b772-6908-407a-994c-845536ac96f7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:35.677599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3b772-6908-407a-994c-845536ac96f7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:35.678046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3b772-6908-407a-994c-845536ac96f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:35.717167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8ad3b772-6908-407a-994c-845536ac96f7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:35.721814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8ad3b772-6908-407a-994c-845536ac96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:35.880493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1030/2050] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:35.901169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:35.903574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:36.034758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5b12-ea3f-4772-9db3-c44ab68e7715 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:36.042457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:36.043673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:36.044295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:36.044869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:36.066486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:36.067096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:36.069745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:36.089698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1021/2051] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:36.372874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:38.445788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:38.618304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:38.618602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:38.618976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:38.692775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] MFA Rules not processed for user `f77bc33a9f3e4fad9b533794c82390b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:38.695701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:38.854586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1031/2052] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:38.859918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:38.877822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25d51-4237-4b70-bff5-f362ab58db5d service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:38.890982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:38.891627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:38.894949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:38.894949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:38.894949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:38.896999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:38.914743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:39.044423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1022/2053] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:07:38 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:39.056828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:39.063765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ecd18-0ad3-4c1a-b06e-f6804bec2916 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:39.148071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:39.149100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:39.150194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:39.150841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:39.151810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:39.154589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:39.360124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1032/2054] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:07:39 2022] POST /identity/v3/users => generated 292 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:07:39.364508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:39.377966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e341c-9227-45c6-8669-f19149a23c2c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:39.385853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:39.387046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:39.387635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:39.388688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:39.389660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:39.391859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:39.409518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1023/2055] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:39.425928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:39.436448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931fc3-d51e-47ef-b62e-00addb790e6d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:39.442272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:39.442966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:39.443251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:39.443528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] PATH_INFO: `/v3/projects/53ae99dc3ee0422ca7f01c674cfd823b/users/4b468752119f43e88da9260ce600fb6b/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:39.467630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ae99dc3ee0422ca7f01c674cfd823b, user_id=4b468752119f43e88da9260ce600fb6b, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:39.473462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:39.770854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1033/2056] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:39 2022] PUT /identity/v3/projects/53ae99dc3ee0422ca7f01c674cfd823b/users/4b468752119f43e88da9260ce600fb6b/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:39.777316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:39.798508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee77b9e-68f4-45d2-8506-bf6686db044b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:39.930656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d323f-84d3-4fed-8b47-79ccd700b965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:39.931716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d323f-84d3-4fed-8b47-79ccd700b965 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:39.931716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d323f-84d3-4fed-8b47-79ccd700b965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:39.932026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d323f-84d3-4fed-8b47-79ccd700b965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:39.932741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d323f-84d3-4fed-8b47-79ccd700b965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:39.934531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d323f-84d3-4fed-8b47-79ccd700b965 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:39.954153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1024/2057] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:07:39.957055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:39.976961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00b39-e7fb-42e3-826e-a951d9bf41fe admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:39.984644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:39.986002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:39.987196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:39.988397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] PATH_INFO: `/v3/projects/53ae99dc3ee0422ca7f01c674cfd823b/users/4b468752119f43e88da9260ce600fb6b/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:39.992840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ae99dc3ee0422ca7f01c674cfd823b, user_id=4b468752119f43e88da9260ce600fb6b, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:39.999064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:40.022088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1034/2058] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:07:39 2022] PUT /identity/v3/projects/53ae99dc3ee0422ca7f01c674cfd823b/users/4b468752119f43e88da9260ce600fb6b/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:07:42.372613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:42.691467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:42.822197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:42.822863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:42.823400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:42.860708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] MFA Rules not processed for user `4b468752119f43e88da9260ce600fb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:42.862493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:42.958566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1025/2059] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:07:42.971470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:42.973788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:43.046847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54421e16-09f8-41c9-b0c1-b4aa0bde62cc admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:43.051938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:43.052769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:43.052867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:43.053140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:43.146452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:43.149525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:43.166924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1035/2060] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:45.024774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:45.153309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:47.155820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:47.425314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:47.428402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:47.444063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7a4c9-c10f-4145-8b85-ff5d7675f513 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:47.448498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31862350-4938-4b1a-94c4-2b76f45d464e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:47.449462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862350-4938-4b1a-94c4-2b76f45d464e service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:47.449977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862350-4938-4b1a-94c4-2b76f45d464e service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:47.450428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862350-4938-4b1a-94c4-2b76f45d464e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:47.465705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:47.472149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31862350-4938-4b1a-94c4-2b76f45d464e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:47.476981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31862350-4938-4b1a-94c4-2b76f45d464e service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:47.499183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1026/2061] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:49.262221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:50.290994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:51.719598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:52.154388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:52.748169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:53.125053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:53.239685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:53.496555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:53.587864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:53.947951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:53.993783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.250975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.286127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.702159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.742298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.862999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.896752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:55.148385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:55.279824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:04.030429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:04.032229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:04.137150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59301c-aaf2-45af-81b7-507c3ffc7876 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:04.153114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:04.153114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.153114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.153114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] PATH_INFO: `/v3/users/90a2c5810dcb4a16aa05702da0d28a9e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.154544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a2c5810dcb4a16aa05702da0d28a9e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:04.154544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:04.587919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a2c5810dcb4a16aa05702da0d28a9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.851101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a2c5810dcb4a16aa05702da0d28a9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.874590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a2c5810dcb4a16aa05702da0d28a9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.875779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a2c5810dcb4a16aa05702da0d28a9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.888484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a2c5810dcb4a16aa05702da0d28a9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.899518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1036/2062] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:04 2022] DELETE /identity/v3/users/90a2c5810dcb4a16aa05702da0d28a9e => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:04.905490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.906183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.906183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.940217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:04.943347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.057499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1027/2063] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:05.070068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.073795 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.164850 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:05.181200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbabf8-fb84-42a8-83c9-bb52909d4f23 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.186720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.187773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.188225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.189255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.204162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.207854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.226442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1037/2064] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:05.891023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.894191 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.907943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06f152-2832-4208-bde9-f83ebae3283c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.914676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.916494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.916494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.916494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] PATH_INFO: `/v3/projects/6808847ac5e54d87b314e298acd9fe0d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.917565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6808847ac5e54d87b314e298acd9fe0d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.919688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.122792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6808847ac5e54d87b314e298acd9fe0d', 'request_id': 'req-69b099d9-21ed-4511-9b7a-f20f79aaf39b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.136781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6808847ac5e54d87b314e298acd9fe0d', 'request_id': 'req-69b099d9-21ed-4511-9b7a-f20f79aaf39b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.151478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6808847ac5e54d87b314e298acd9fe0d', 'request_id': 'req-69b099d9-21ed-4511-9b7a-f20f79aaf39b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.304808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] The token cache is being invalidate because project 6808847ac5e54d87b314e298acd9fe0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:06.305321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.306626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.309425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1028/2065] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:05 2022] DELETE /identity/v3/projects/6808847ac5e54d87b314e298acd9fe0d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:06.317544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.328526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0c58c-e1df-41fe-a7df-91e7a3cb2e4c service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.417847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.418782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.419256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.419542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] PATH_INFO: `/v3/users/b2614bb3b7f441c498eca17977814497` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.421347 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2614bb3b7f441c498eca17977814497)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.425427 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.800951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2614bb3b7f441c498eca17977814497'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.936454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2614bb3b7f441c498eca17977814497'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.952873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2614bb3b7f441c498eca17977814497'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.953722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2614bb3b7f441c498eca17977814497'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.963477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4bf73d88-3f98-47d5-ba27-9be4108b0ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2614bb3b7f441c498eca17977814497'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.971088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1038/2066] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:06 2022] DELETE /identity/v3/users/b2614bb3b7f441c498eca17977814497 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:08.445995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.448555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:08.467306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b099d9-21ed-4511-9b7a-f20f79aaf39b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.548623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.549462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.549960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.550725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] PATH_INFO: `/v3/projects/322bec89bbe04a4d9de85fd35d7b1338` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.551736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=322bec89bbe04a4d9de85fd35d7b1338)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.553889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.590882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:08.741832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322bec89bbe04a4d9de85fd35d7b1338', 'request_id': 'req-acb2d911-1a90-445c-8618-143fef0db5ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.750210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322bec89bbe04a4d9de85fd35d7b1338', 'request_id': 'req-acb2d911-1a90-445c-8618-143fef0db5ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.760923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322bec89bbe04a4d9de85fd35d7b1338', 'request_id': 'req-acb2d911-1a90-445c-8618-143fef0db5ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.883731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] The token cache is being invalidate because project 322bec89bbe04a4d9de85fd35d7b1338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:08.884205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.885414 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acb2d911-1a90-445c-8618-143fef0db5ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.896289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.896836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.899009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.914008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1029/2067] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:08 2022] DELETE /identity/v3/projects/322bec89bbe04a4d9de85fd35d7b1338 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:08.966962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:08.968880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:09.102873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1039/2068] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:09.119945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.120425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.120929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.175753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:09.178266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:09.211084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1030/2069] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:09.216905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.231009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:09.242602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66c326-63af-4b52-8254-02ab9273b29e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.273054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.274086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.275096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.275604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.277290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:09.278114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.281281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.311630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1040/2070] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:09.313514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.332135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606762d-8a9e-4184-a0a0-a787ec51cfbf None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.341127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.342268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.342783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.343292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.344611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.349777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.583964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1031/2071] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:09 2022] POST /identity/v3/projects => generated 387 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:09.585618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.606626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd30589-8dd7-4e57-98e2-241612951190 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.750045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.750792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.751141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.751616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.753504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.756451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.995781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1041/2072] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:09 2022] POST /identity/v3/users => generated 294 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:10.000161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.012348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee5cbf-e397-4eb3-af30-9a688f25a22c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.018738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.019975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.020255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.020697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.021204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.022870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.034212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1032/2073] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:10.039453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.053833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d468cb-56b0-4266-887c-7c6d0120ca38 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.062284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fa649-700b-44f3-9035-f34545fcbc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.063153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fa649-700b-44f3-9035-f34545fcbc1d admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.063617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fa649-700b-44f3-9035-f34545fcbc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.063905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fa649-700b-44f3-9035-f34545fcbc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.065470 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fa649-700b-44f3-9035-f34545fcbc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.067184 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fa649-700b-44f3-9035-f34545fcbc1d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.083728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1042/2074] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:10.099036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.115101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5eaa35-e4b4-499f-92a9-2d199ee4f6b7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.122104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.122591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.122874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.123154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] PATH_INFO: `/v3/projects/171f46f60e904def83d051c9415b6b7e/users/f22bbd320e3e45a8857895ba973a9528/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.155638 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=171f46f60e904def83d051c9415b6b7e, user_id=f22bbd320e3e45a8857895ba973a9528, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.159849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.314136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1033/2075] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:10 2022] PUT /identity/v3/projects/171f46f60e904def83d051c9415b6b7e/users/f22bbd320e3e45a8857895ba973a9528/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:10.327182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca8dcf-1e42-4ffc-90dc-ffa2225c6f33 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.327182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca8dcf-1e42-4ffc-90dc-ffa2225c6f33 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.327468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca8dcf-1e42-4ffc-90dc-ffa2225c6f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.385634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2dca8dcf-1e42-4ffc-90dc-ffa2225c6f33 None None] MFA Rules not processed for user `f22bbd320e3e45a8857895ba973a9528`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:10.388003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2dca8dcf-1e42-4ffc-90dc-ffa2225c6f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.552865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1043/2076] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:10.560033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.680760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9c43-4b81-40fb-b017-df8debbabac2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.688336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.689128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.689573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.690120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.691319 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.693704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.896776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1034/2077] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/projects => generated 387 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:10.901767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.902228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.903387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.903691 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.932002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:10.932002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.984523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:11.037979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1044/2078] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:11.039055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66de255-d446-41b5-96b1-73b6d4aeb67f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.044890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.045533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.045827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.046254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.047688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.050228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.103744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.104807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:11.111158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:11.196936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1035/2079] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/users => generated 300 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:11.200531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.235117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e0666-7309-4766-91bf-4bc553e0d47d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.250137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.261313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.261313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.261313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.261313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.261842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.286215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1036/2080] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:11.291043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.291631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ee0e3-5579-4915-8055-96c2c1fba06d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.310099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.311012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.311501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.311962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.313649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fa6dc-6e6f-4ae1-9d5d-982cdf501cd8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.319710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.320406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.320689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.320971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] PATH_INFO: `/v3/projects/8777a655b4ac4874ab7aacda5041ca68/users/642a90f7ec3543cc9768650f134a5f47/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.323599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.325571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.342135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1045/2081] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:08:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:08:11.348099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8777a655b4ac4874ab7aacda5041ca68, user_id=642a90f7ec3543cc9768650f134a5f47, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.354073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.560533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1037/2082] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:11 2022] PUT /identity/v3/projects/8777a655b4ac4874ab7aacda5041ca68/users/642a90f7ec3543cc9768650f134a5f47/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:11.571661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.594905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b187acd-4326-46b9-a02b-d7aebd05dbbc service swift] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.736289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceba696-be0f-4091-8bb2-b3e9e345460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.737966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceba696-be0f-4091-8bb2-b3e9e345460c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.738742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceba696-be0f-4091-8bb2-b3e9e345460c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.739488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceba696-be0f-4091-8bb2-b3e9e345460c admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.741015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceba696-be0f-4091-8bb2-b3e9e345460c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.745186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceba696-be0f-4091-8bb2-b3e9e345460c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.773890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1046/2083] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:11.774817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.787517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ef864-dacd-4115-a6b6-3042c0761c5b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.793077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.793844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.794278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.794639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] PATH_INFO: `/v3/projects/8777a655b4ac4874ab7aacda5041ca68/users/642a90f7ec3543cc9768650f134a5f47/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.796814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8777a655b4ac4874ab7aacda5041ca68, user_id=642a90f7ec3543cc9768650f134a5f47, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.799546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.814200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1038/2084] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:11 2022] PUT /identity/v3/projects/8777a655b4ac4874ab7aacda5041ca68/users/642a90f7ec3543cc9768650f134a5f47/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:11.825929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.826223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.826505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.885229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] MFA Rules not processed for user `642a90f7ec3543cc9768650f134a5f47`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:11.891800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:12.013755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1047/2085] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:12.383270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:12.385616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:12.475039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae743652-d9a7-4c41-a438-a5a5ea4a5caa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:12.483835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bad33-4c0c-4633-b869-fbf54dddbf88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:12.484520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bad33-4c0c-4633-b869-fbf54dddbf88 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:12.484804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bad33-4c0c-4633-b869-fbf54dddbf88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:12.485081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bad33-4c0c-4633-b869-fbf54dddbf88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:12.497281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bad33-4c0c-4633-b869-fbf54dddbf88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:12.498909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bad33-4c0c-4633-b869-fbf54dddbf88 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:12.510584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1039/2086] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:12.549128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:12.603434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:12.696136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93e48c-2480-4eb2-90da-04658b2c3234 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:12.701751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:12.702552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:12.702995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:12.703285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] PATH_INFO: `/v3/users/f22bbd320e3e45a8857895ba973a9528` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:12.704482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22bbd320e3e45a8857895ba973a9528)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:12.707425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:13.070382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:13.070916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:13.072374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:13.094442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22bbd320e3e45a8857895ba973a9528'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.101667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:13.103036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:13.176489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1040/2087] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:13.188629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22bbd320e3e45a8857895ba973a9528'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.199536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:13.214877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22bbd320e3e45a8857895ba973a9528'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.215792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22bbd320e3e45a8857895ba973a9528'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.230932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c48c19bd-ae06-44d1-aa56-159a7112b103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22bbd320e3e45a8857895ba973a9528'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.232023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:13.243105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1048/2088] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:12 2022] DELETE /identity/v3/users/f22bbd320e3e45a8857895ba973a9528 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:13.250865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:13.252640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:13.252979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:13.278175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:13.279422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:13.347672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce46ac-9d09-46e7-9bd2-5850e1466328 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:13.358256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1049/2089] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:13.370587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:13.372128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:13.428532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:13.448869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1110e55-5c97-4043-901f-ad3b4261ffcf None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:13.453435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:13.454089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:13.454374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:13.454702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:13.467015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:13.468869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:13.470473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:13.471163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:13.471448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:13.471731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:13.474229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:13.482949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1050/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:13.556864 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:13.558787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:13.584422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1041/2091] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:08:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:08:14.107827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:14.109796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:14.125268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10c6c1-9d61-4802-8ddf-5171215b67d3 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:14.132731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:14.133497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.133776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.134289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] PATH_INFO: `/v3/projects/171f46f60e904def83d051c9415b6b7e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.135615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=171f46f60e904def83d051c9415b6b7e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:14.137773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:14.306234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171f46f60e904def83d051c9415b6b7e', 'request_id': 'req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.317998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171f46f60e904def83d051c9415b6b7e', 'request_id': 'req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.429135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171f46f60e904def83d051c9415b6b7e', 'request_id': 'req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.454645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] The token cache is being invalidate because project 171f46f60e904def83d051c9415b6b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:14.455144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.457998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a82aa3da-11c9-46e8-9f1d-7f8e26c2dfcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.461831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1051/2092] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:14 2022] DELETE /identity/v3/projects/171f46f60e904def83d051c9415b6b7e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:14.470457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:14.484303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7894d3-de98-426a-bd92-82db04fedef1 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:14.608657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:14.611328 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.612151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.612844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] PATH_INFO: `/v3/users/642a90f7ec3543cc9768650f134a5f47` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.615207 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=642a90f7ec3543cc9768650f134a5f47)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:14.619563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:14.623516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:14.698798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.699312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.699889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.725891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:14.726965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:14.816352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1052/2093] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:08:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:14.832479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:14.834224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:14.883357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642a90f7ec3543cc9768650f134a5f47'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.885057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642a90f7ec3543cc9768650f134a5f47'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.916178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642a90f7ec3543cc9768650f134a5f47'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.949447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a9d8b-cf2e-42aa-960c-128636b75c93 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:14.955208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:14.956204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.956554 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.956835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.966522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:14.971326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:14.979072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:14.993217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1053/2094] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:08:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:08:15.018269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642a90f7ec3543cc9768650f134a5f47'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.028326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642a90f7ec3543cc9768650f134a5f47'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.042964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1042/2095] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:14 2022] DELETE /identity/v3/users/642a90f7ec3543cc9768650f134a5f47 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:15.549254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:15.551461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:15.572831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584065c-4ff5-4347-bdad-3be0c911c51a service swift] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:15.639216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:15.640951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:15.646376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:15.647286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.647748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.648178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] PATH_INFO: `/v3/users/d6aa518cc8004be6b79c82623574b056` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.649634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6aa518cc8004be6b79c82623574b056)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:15.652152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:15.665325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e7b8-84b8-4db5-871c-6dfa040e1ade admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:15.690717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:15.694070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.694577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.695018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] PATH_INFO: `/v3/projects/8777a655b4ac4874ab7aacda5041ca68` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.696802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8777a655b4ac4874ab7aacda5041ca68)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:15.699736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.028666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6aa518cc8004be6b79c82623574b056'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.029931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8777a655b4ac4874ab7aacda5041ca68', 'request_id': 'req-244cf16a-071b-431d-9d98-6b3a32c45f4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.041303 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8777a655b4ac4874ab7aacda5041ca68', 'request_id': 'req-244cf16a-071b-431d-9d98-6b3a32c45f4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.052644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8777a655b4ac4874ab7aacda5041ca68', 'request_id': 'req-244cf16a-071b-431d-9d98-6b3a32c45f4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.108086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6aa518cc8004be6b79c82623574b056'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.127959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6aa518cc8004be6b79c82623574b056'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.128944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6aa518cc8004be6b79c82623574b056'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.139467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fdbb2939-d451-4ddd-9f0a-e978ba272503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6aa518cc8004be6b79c82623574b056'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.148921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1054/2096] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:15 2022] DELETE /identity/v3/users/d6aa518cc8004be6b79c82623574b056 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:16.154133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb854-d12a-4a28-9de0-9d1857835517 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.154133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb854-d12a-4a28-9de0-9d1857835517 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.154133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb854-d12a-4a28-9de0-9d1857835517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.182687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b95eb854-d12a-4a28-9de0-9d1857835517 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:16.183249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] The token cache is being invalidate because project 8777a655b4ac4874ab7aacda5041ca68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:16.183693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.184771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b95eb854-d12a-4a28-9de0-9d1857835517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.184968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-244cf16a-071b-431d-9d98-6b3a32c45f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:16.187793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1043/2097] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:15 2022] DELETE /identity/v3/projects/8777a655b4ac4874ab7aacda5041ca68 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:16.190273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.190800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.191313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.218999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:16.221446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.309661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1055/2098] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:16.315432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.315910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.316478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.350769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1044/2099] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:16.352090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.354171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.361060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:16.361060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.380387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:16.387062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1056/2100] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:16.389125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.395341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.420117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:16.491633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0aeb6-1df8-4639-8397-541df4a941df None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.499404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a574ec-1152-4387-8495-232834fadbd1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.500014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a574ec-1152-4387-8495-232834fadbd1 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.500307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a574ec-1152-4387-8495-232834fadbd1 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.500602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a574ec-1152-4387-8495-232834fadbd1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.511926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a574ec-1152-4387-8495-232834fadbd1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.515368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a574ec-1152-4387-8495-232834fadbd1 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.522660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd17caa-c87c-44f7-996f-303be69323dd None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.532823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.535278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1045/2101] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:08:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:08:16.535630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.535630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.535630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.540763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.541274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.541971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.558568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.558568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.574022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1057/2102] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:16.582628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:16.584928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.607678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1046/2103] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:16.612894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.632475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d055bd-cef0-4150-ab22-b2907ffeacdf service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.635988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.636631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.636956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.637236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.649506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:16.649963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.656368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.671920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1058/2104] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:16.678008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.700202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e375-2ccc-4a93-8545-f3a37635ef5c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.727948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.728711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.729095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.729527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.730339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.732432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.920079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1047/2105] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/projects => generated 381 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:16.924802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.937309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b6e1c-165c-4067-9335-852b365d9d9e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.042778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.043531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.043875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.044387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.045031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.047142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.520718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1059/2106] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/users => generated 291 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:17.535035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.551867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79308-6fae-4991-9347-edd4c1350288 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.560145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.561258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.561756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.562376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.563616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.566934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.579405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1048/2107] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:17.592391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.604106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306a24a-5248-4e7a-bf18-224e86afabd1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.609025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.609685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.610030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.610328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.611008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.613679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.626610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1060/2108] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:17.631799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.642503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b73ef-736b-4574-9b8b-bc6464ff50a4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.648273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc5d00-aae4-4cac-ac30-0c4661031070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.648990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc5d00-aae4-4cac-ac30-0c4661031070 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.649271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc5d00-aae4-4cac-ac30-0c4661031070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.649548 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc5d00-aae4-4cac-ac30-0c4661031070 admin admin] PATH_INFO: `/v3/projects/48a6cd393fcf4c68af3db7d4c457d6c2/users/63aa6b25b5744fffb2f8ae22fa06bf32/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.666993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc5d00-aae4-4cac-ac30-0c4661031070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a6cd393fcf4c68af3db7d4c457d6c2, user_id=63aa6b25b5744fffb2f8ae22fa06bf32, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.669662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc5d00-aae4-4cac-ac30-0c4661031070 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.812607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.814813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:17.843746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cb30e-7988-47ad-9bb7-5fc63456160d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.843851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1049/2109] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:17 2022] PUT /identity/v3/projects/48a6cd393fcf4c68af3db7d4c457d6c2/users/63aa6b25b5744fffb2f8ae22fa06bf32/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:17.855719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.855841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.856055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.929473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.930211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.930619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.930897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] PATH_INFO: `/v3/projects/e0f123b8ea4747f7aae0ddc146019cd6` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.934811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f123b8ea4747f7aae0ddc146019cd6)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.936854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.962362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] MFA Rules not processed for user `63aa6b25b5744fffb2f8ae22fa06bf32`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:17.962362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.113539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1050/2110] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:18.119658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.156885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baee26-75e8-497f-aea4-636f7165cf1c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.165118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.165830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.166148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.166449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.167773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.169761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.245553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f123b8ea4747f7aae0ddc146019cd6', 'request_id': 'req-0130b413-e09a-4cb2-8e19-e1e0b6f64405'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.254287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f123b8ea4747f7aae0ddc146019cd6', 'request_id': 'req-0130b413-e09a-4cb2-8e19-e1e0b6f64405'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.301201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1051/2111] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:18.301748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.393800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f123b8ea4747f7aae0ddc146019cd6', 'request_id': 'req-0130b413-e09a-4cb2-8e19-e1e0b6f64405'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.433421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21c1ad-ee88-4182-b212-ecc4cc4cf7e8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.437760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] The token cache is being invalidate because project e0f123b8ea4747f7aae0ddc146019cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:18.445224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.449314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0130b413-e09a-4cb2-8e19-e1e0b6f64405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.450391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1061/2112] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:17 2022] DELETE /identity/v3/projects/e0f123b8ea4747f7aae0ddc146019cd6 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:18.459720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fbb4f-3cf8-4ed0-9ffc-8ffbe1e5f5c3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.460267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.460684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fbb4f-3cf8-4ed0-9ffc-8ffbe1e5f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.461260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.461547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.461868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.462266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fbb4f-3cf8-4ed0-9ffc-8ffbe1e5f5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.462557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.466482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.521544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:18.533855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-067fbb4f-3cf8-4ed0-9ffc-8ffbe1e5f5c3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:18.535243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-067fbb4f-3cf8-4ed0-9ffc-8ffbe1e5f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.565725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1062/2113] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:18.574227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.574895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.575176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.613003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:18.615746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.627738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1063/2114] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:18.632314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.634742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1052/2115] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/users => generated 297 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:18.649571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71764-70bd-4b08-8887-7d73788bb20d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.652208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.654071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.656830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.657629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.658129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.658572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.659511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:18.660042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.662337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.678998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1064/2116] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:18.683420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.692555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97b732-9dca-4831-b257-14df29874fdc admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.706832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.709276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.710346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.716767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b575-e326-46a4-b72a-15e9f4057ba1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.718637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.719359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.721123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:18.721484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.729341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.731481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.732201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.732965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.734828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.739445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.748835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1053/2117] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:18.754231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.770005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2ff4-448d-479a-999d-f2068513db5c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.781442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.782172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.782461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.782740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] PATH_INFO: `/v3/projects/8fea2773d99e4d0c80df710daeaabbed/users/0afc82f863bb46e0913f7390d56db473/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.820696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fea2773d99e4d0c80df710daeaabbed, user_id=0afc82f863bb46e0913f7390d56db473, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.827888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.917160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1065/2118] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:18.926191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.944745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1054/2119] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:18 2022] PUT /identity/v3/projects/8fea2773d99e4d0c80df710daeaabbed/users/0afc82f863bb46e0913f7390d56db473/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:18.951124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.091185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52587-749a-4c88-ac42-fb4dfe696989 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.141466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3f72-f66f-407a-a687-cb33568dcecc admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.156857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.157579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.157888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.158235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.159560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.161381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.191609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1055/2120] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:19.191609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.191609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:19.208842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa39605-9715-476c-9e96-e844cf91aea4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.216757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.217613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.220131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.220131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] PATH_INFO: `/v3/users/f77bc33a9f3e4fad9b533794c82390b5` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.220131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77bc33a9f3e4fad9b533794c82390b5)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.223171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.256159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.257315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.257880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.258418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.260429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.265134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.536180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:19.543874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77bc33a9f3e4fad9b533794c82390b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.545034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77bc33a9f3e4fad9b533794c82390b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.568079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1066/2121] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/users => generated 293 bytes in 643 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:19.571527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.572274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77bc33a9f3e4fad9b533794c82390b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.591473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b100-923c-41f4-8807-a240cabc32a4 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.608271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.609008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.609574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.610095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] PATH_INFO: `/v3/projects/8fea2773d99e4d0c80df710daeaabbed/users/0afc82f863bb46e0913f7390d56db473/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.615522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fea2773d99e4d0c80df710daeaabbed, user_id=0afc82f863bb46e0913f7390d56db473, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.617508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.642416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1067/2122] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:19 2022] PUT /identity/v3/projects/8fea2773d99e4d0c80df710daeaabbed/users/0afc82f863bb46e0913f7390d56db473/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:19.642416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.719809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77bc33a9f3e4fad9b533794c82390b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.729193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d851571-e4d4-4b3e-afd4-113a00c6e31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77bc33a9f3e4fad9b533794c82390b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.740491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1056/2123] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:19 2022] DELETE /identity/v3/users/f77bc33a9f3e4fad9b533794c82390b5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:19.740491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.740491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.740491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.741465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb6de1-9a6c-44f6-a36e-3f5a41643ba1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.779052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] MFA Rules not processed for user `0afc82f863bb46e0913f7390d56db473`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:19.780462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:19.835091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.835737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.836039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.836330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.836993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.838760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.850960 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1068/2124] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:19.857046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.858700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:19.872369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4538d56-823c-44dc-8a9d-cf21ba5e0263 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.873388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1057/2125] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:19.881847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.889082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.889082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.889082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.889082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.889082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.889612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.889612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:19.900653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1069/2126] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:19.904994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.919038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6375-d2df-4266-b022-785067ca8dca admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.926266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37231465-b616-4e2c-b962-f38f63acd0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.927368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231465-b616-4e2c-b962-f38f63acd0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.928418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231465-b616-4e2c-b962-f38f63acd0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.928907 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231465-b616-4e2c-b962-f38f63acd0b4 admin admin] PATH_INFO: `/v3/projects/28b80cc1383e4c28bcc7a3699b9e2ad9/users/f5896d8a685940c1bbba6b7dd86f3d1b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.953857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37231465-b616-4e2c-b962-f38f63acd0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b80cc1383e4c28bcc7a3699b9e2ad9, user_id=f5896d8a685940c1bbba6b7dd86f3d1b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.958628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37231465-b616-4e2c-b962-f38f63acd0b4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.974054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ccad7-0ca9-4caf-a496-41cd4e656c6e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.978887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.978887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.979174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.979393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.990272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.992622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:20.008965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1058/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:20.084706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1070/2128] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:19 2022] PUT /identity/v3/projects/28b80cc1383e4c28bcc7a3699b9e2ad9/users/f5896d8a685940c1bbba6b7dd86f3d1b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:20.086949 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:20.090168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:20.111617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.111617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.111617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.154874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f056c3-30a1-4fd1-ba7b-ceae885932bb service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:20.156216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] MFA Rules not processed for user `f5896d8a685940c1bbba6b7dd86f3d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:20.157815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:20.230031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:20.266947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:20.268387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.268712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.269047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] PATH_INFO: `/v3/projects/404bac8465994797b72d704bba719f31` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.270601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=404bac8465994797b72d704bba719f31)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:20.277491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:20.278937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1071/2129] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:20.296714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:20.383524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cd21e-a373-483a-80be-75299c1c1671 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:20.387788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:20.388513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.388869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.389214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.405061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:20.406563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:20.419215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1072/2130] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:20.460193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '404bac8465994797b72d704bba719f31', 'request_id': 'req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.469551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '404bac8465994797b72d704bba719f31', 'request_id': 'req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.482262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '404bac8465994797b72d704bba719f31', 'request_id': 'req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.555596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:20.591148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] The token cache is being invalidate because project 404bac8465994797b72d704bba719f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:20.591756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.592645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd7ff1a1-e4e1-4052-b8e8-8b17d435ab70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.594284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1059/2131] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:20 2022] DELETE /identity/v3/projects/404bac8465994797b72d704bba719f31 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:20.604676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:20.616950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfb033-c537-4f98-96cb-f58fe34ad67b service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:20.703284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:20.704005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.704192 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.704469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] PATH_INFO: `/v3/users/4b468752119f43e88da9260ce600fb6b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.705609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b468752119f43e88da9260ce600fb6b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:20.708104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.217126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b468752119f43e88da9260ce600fb6b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.235240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.235756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.235851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.256642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:21.265808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.320057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1060/2132] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:08:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:21.336360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.337846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.349020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b468752119f43e88da9260ce600fb6b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.374810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b468752119f43e88da9260ce600fb6b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.376154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b468752119f43e88da9260ce600fb6b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.386317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b468752119f43e88da9260ce600fb6b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.395640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1073/2133] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:20 2022] DELETE /identity/v3/users/4b468752119f43e88da9260ce600fb6b => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:21.451961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ebac3-97ee-42ec-8295-3d663d16c7ed None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.541568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.542760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.543286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.543801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.547203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.569334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:21.645817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.647603 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.666503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1061/2134] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:08:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:08:21.796325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.799259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.836031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.838565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.983832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aea8-00a1-4347-a562-32b5691f01c8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.990049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.990552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.990993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.991283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] PATH_INFO: `/v3/users/446af6bea953401ead7e8a9c9604d8d0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.992145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7421ae5-906e-4b1e-9829-08ba3e7bcf37 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.992958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=446af6bea953401ead7e8a9c9604d8d0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.995025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.000079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.000891 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.001279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.001665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] PATH_INFO: `/v3/projects/53ae99dc3ee0422ca7f01c674cfd823b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.004084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ae99dc3ee0422ca7f01c674cfd823b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.006430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.376385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446af6bea953401ead7e8a9c9604d8d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.388158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ae99dc3ee0422ca7f01c674cfd823b', 'request_id': 'req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.405197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ae99dc3ee0422ca7f01c674cfd823b', 'request_id': 'req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.417982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ae99dc3ee0422ca7f01c674cfd823b', 'request_id': 'req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.480497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446af6bea953401ead7e8a9c9604d8d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.501107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446af6bea953401ead7e8a9c9604d8d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.502663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446af6bea953401ead7e8a9c9604d8d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.521281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446af6bea953401ead7e8a9c9604d8d0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.533113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1074/2135] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:21 2022] DELETE /identity/v3/users/446af6bea953401ead7e8a9c9604d8d0 => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:22.533113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.535208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.737682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67dfa2-e1e0-4857-8ece-926113b9c063 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.746058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0afc82f863bb46e0913f7390d56db473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fea2773d99e4d0c80df710daeaabbed', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.746975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.747366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.747819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.748696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:22.749117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.751144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e214e4-59cc-4a5d-a46f-358fce6ed0eb tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.774553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] The token cache is being invalidate because project 53ae99dc3ee0422ca7f01c674cfd823b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:22.780362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.780362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e33deb7a-a935-4438-9cd5-7a1e5602ff7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:22.781179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1062/2136] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:21 2022] DELETE /identity/v3/projects/53ae99dc3ee0422ca7f01c674cfd823b => generated 0 bytes in 946 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:22.782705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.784350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.784751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.799867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1075/2137] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:22.799867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.799867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.799867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.839152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:22.841296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.852441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:22.855091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.934297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1063/2138] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:22.935320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.939417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.960515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a324b12-15fa-44ab-ab45-c958c2466e99 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.969048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032d63f-2499-4e17-9971-3fc42488c545 tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0afc82f863bb46e0913f7390d56db473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fea2773d99e4d0c80df710daeaabbed', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.969048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d63f-2499-4e17-9971-3fc42488c545 tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.969048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d63f-2499-4e17-9971-3fc42488c545 tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.970044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d63f-2499-4e17-9971-3fc42488c545 tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.970883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032d63f-2499-4e17-9971-3fc42488c545 tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.974169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032d63f-2499-4e17-9971-3fc42488c545 tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.040272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:23.046974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1076/2139] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:23.049323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.050671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:23.102133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1064/2140] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/projects => generated 355 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:23.107097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fc872-66da-4b06-9544-69d2f808503a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.107426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fc872-66da-4b06-9544-69d2f808503a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.107752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fc872-66da-4b06-9544-69d2f808503a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.129838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:23.154465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-329fc872-66da-4b06-9544-69d2f808503a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:23.156503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-329fc872-66da-4b06-9544-69d2f808503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:23.203968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fc11-a15c-4e2e-9cd6-fa13ddd66c56 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.310146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1065/2141] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:23.310146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fc872-66da-4b06-9544-69d2f808503a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.352334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.355067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.355067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.355067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.380429 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.382749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.397098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1077/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:23.398613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.413504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f0ec-14cb-4a4e-aab3-299c710317f3 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.433483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.435051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.435666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.438214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.440553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:23.441603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.459555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.512508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1078/2143] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:23.516016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.520031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fc872-66da-4b06-9544-69d2f808503a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.553599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0afc82f863bb46e0913f7390d56db473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fea2773d99e4d0c80df710daeaabbed', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.557808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.558533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.559716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.560833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:23.562232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.564436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.570690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44d54b-c3e4-46ef-a83d-dfd3a2e91321 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.586964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1066/2144] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:23.599262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.601073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.603722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.604266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.611793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.618775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.681832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.698798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce3222-6899-45a3-a73f-1acd86e6477d tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.711955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0afc82f863bb46e0913f7390d56db473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fea2773d99e4d0c80df710daeaabbed', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.712978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.714152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.714543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] PATH_INFO: `/v3/projects/02583663fd4546a5beced722fb91b788` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.716472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorizing `identity:delete_project(project_id=02583663fd4546a5beced722fb91b788)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.718810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.800715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1079/2145] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:23 2022] POST /identity/v3/projects => generated 361 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:23.801811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.906540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb93d2-e423-4302-98cb-becee13d166b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.912484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.913385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.913831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.914578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.915424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.917622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.968244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02583663fd4546a5beced722fb91b788', 'request_id': 'req-60c7333c-fd33-4f12-901a-47ddc5b2d94f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:23.981269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02583663fd4546a5beced722fb91b788', 'request_id': 'req-60c7333c-fd33-4f12-901a-47ddc5b2d94f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:23.992110 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02583663fd4546a5beced722fb91b788', 'request_id': 'req-60c7333c-fd33-4f12-901a-47ddc5b2d94f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.090064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1080/2146] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:23 2022] POST /identity/v3/users => generated 281 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:24.102706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:24.141722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] The token cache is being invalidate because project 02583663fd4546a5beced722fb91b788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:24.141722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.141722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.143429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1067/2147] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:23 2022] DELETE /identity/v3/projects/02583663fd4546a5beced722fb91b788 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:24.145241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:24.150723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf2f5d-6f73-4651-8e18-95efe958146e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:24.173566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:24.317436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:24.319174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.321466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.322623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.324493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:24.329770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:24.349713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1081/2148] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:24.355931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:24.358909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:24.380418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bac22-63d7-4b8e-9425-f64effb7514f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:24.390418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:24.391777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.392213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.393091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.394108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:24.396792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:24.402818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7333c-fd33-4f12-901a-47ddc5b2d94f tempest-VolumeQuotaClassesTest-1490249842 tempest-VolumeQuotaClassesTest-1490249842-project-admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:24.410557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:24.411811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.412306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.412767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] PATH_INFO: `/v3/projects/1c13f664f15145fdb9b3c4b9f8496d80` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.414168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c13f664f15145fdb9b3c4b9f8496d80)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:24.417331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:24.438710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1082/2149] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:24.442788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:24.481053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e274180-899e-4662-9a12-5a366aeebae1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:24.489359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eef071-4356-4ac7-97c2-fe2092c74a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:24.490092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eef071-4356-4ac7-97c2-fe2092c74a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.490433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eef071-4356-4ac7-97c2-fe2092c74a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.490674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eef071-4356-4ac7-97c2-fe2092c74a1f admin admin] PATH_INFO: `/v3/projects/b8fe9373326148b4948ef6927950659f/users/092354076c4b485b824055a99b4363ff/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.512928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eef071-4356-4ac7-97c2-fe2092c74a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fe9373326148b4948ef6927950659f, user_id=092354076c4b485b824055a99b4363ff, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:24.516211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eef071-4356-4ac7-97c2-fe2092c74a1f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:24.684234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1083/2150] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:24 2022] PUT /identity/v3/projects/b8fe9373326148b4948ef6927950659f/users/092354076c4b485b824055a99b4363ff/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:24.696123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.697097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.697588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.703416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c13f664f15145fdb9b3c4b9f8496d80', 'request_id': 'req-c285cfc4-e392-4921-b32a-a112e32a80a1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.712665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c13f664f15145fdb9b3c4b9f8496d80', 'request_id': 'req-c285cfc4-e392-4921-b32a-a112e32a80a1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.719962 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c13f664f15145fdb9b3c4b9f8496d80', 'request_id': 'req-c285cfc4-e392-4921-b32a-a112e32a80a1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.812904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] MFA Rules not processed for user `092354076c4b485b824055a99b4363ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:24.814207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:24.835238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] The token cache is being invalidate because project 1c13f664f15145fdb9b3c4b9f8496d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:24.835680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.837101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c285cfc4-e392-4921-b32a-a112e32a80a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.838939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1068/2151] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:24 2022] DELETE /identity/v3/projects/1c13f664f15145fdb9b3c4b9f8496d80 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:24.847310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c117302-4eec-462a-9252-e5b1d091e4ca None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.849953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c117302-4eec-462a-9252-e5b1d091e4ca None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.850398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c117302-4eec-462a-9252-e5b1d091e4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.886763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4c117302-4eec-462a-9252-e5b1d091e4ca None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:24.888463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4c117302-4eec-462a-9252-e5b1d091e4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:24.919211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1084/2152] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:25.005801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.008514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.025914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1069/2153] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:25.028130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.028525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.028917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.044620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08731e-f5d5-418a-ae95-1cb28a45b557 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.063436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.064957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.065931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.066574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] PATH_INFO: `/v3/users/f5896d8a685940c1bbba6b7dd86f3d1b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.072160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5896d8a685940c1bbba6b7dd86f3d1b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.092126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.115112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:25.117262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.140942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1070/2154] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:25.142852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.164348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda41ed-711d-4d49-8674-8293f9a30c74 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.186036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.191118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.191636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.197534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.205555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:25.206493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.223197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.256667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:25.274399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1071/2155] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:25.276574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.293646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.344543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:25.433405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8410fa-5c56-4fc3-8eec-558692843f10 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.446953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.448692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.449538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.450337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.478938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5896d8a685940c1bbba6b7dd86f3d1b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:25.489373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.492878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.537335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1072/2156] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:25.543914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.587609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f381f9f-199e-48aa-a975-87ddf6c1caea service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.588456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5896d8a685940c1bbba6b7dd86f3d1b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:25.599887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.601642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.602520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.604101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.608281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.612695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.615585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5896d8a685940c1bbba6b7dd86f3d1b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:25.619012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5896d8a685940c1bbba6b7dd86f3d1b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:25.642426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5896d8a685940c1bbba6b7dd86f3d1b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:25.652949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1085/2157] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:25 2022] DELETE /identity/v3/users/f5896d8a685940c1bbba6b7dd86f3d1b => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:25.654993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.657275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.795608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7509bd2-844b-420f-8677-fd151619dcc3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.803894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.804829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.805279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.805609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] PATH_INFO: `/v3/users/63aa6b25b5744fffb2f8ae22fa06bf32` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.806948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63aa6b25b5744fffb2f8ae22fa06bf32)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.808949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.811667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1073/2158] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:25 2022] POST /identity/v3/projects => generated 391 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:25.814766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.817119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.905682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7bac1-a190-4912-a136-19f0bb8b8a1e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.913351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.914043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.914332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.914607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] PATH_INFO: `/v3/users/092354076c4b485b824055a99b4363ff` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.915815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=092354076c4b485b824055a99b4363ff)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.917940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:26.142342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63aa6b25b5744fffb2f8ae22fa06bf32'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.243426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092354076c4b485b824055a99b4363ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.244088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092354076c4b485b824055a99b4363ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.264554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092354076c4b485b824055a99b4363ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.310457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63aa6b25b5744fffb2f8ae22fa06bf32'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.337803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63aa6b25b5744fffb2f8ae22fa06bf32'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.338731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63aa6b25b5744fffb2f8ae22fa06bf32'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.380195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3587c671-b83a-4199-afb2-2c1e6a93ea5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63aa6b25b5744fffb2f8ae22fa06bf32'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.380195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1086/2159] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:25 2022] DELETE /identity/v3/users/63aa6b25b5744fffb2f8ae22fa06bf32 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:26.380195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.380195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.380195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.395298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:26.396444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.472112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1087/2160] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:26.472998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:26.474346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.492892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cd47a-e0c3-4cf0-81f7-a9c126cf8132 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:26.497735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b3d8-15d0-4eb4-8297-23a9ea888909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:26.498471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b3d8-15d0-4eb4-8297-23a9ea888909 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.498750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b3d8-15d0-4eb4-8297-23a9ea888909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.499023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b3d8-15d0-4eb4-8297-23a9ea888909 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.499672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b3d8-15d0-4eb4-8297-23a9ea888909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:26.501630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b3d8-15d0-4eb4-8297-23a9ea888909 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:26.767046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092354076c4b485b824055a99b4363ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.774916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1088/2161] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:26 2022] POST /identity/v3/users => generated 296 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:26.776525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.776974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.777573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.787174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092354076c4b485b824055a99b4363ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.796905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1074/2162] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:25 2022] DELETE /identity/v3/users/092354076c4b485b824055a99b4363ff => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:26.800905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:26.802157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:26.802292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.803574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.888720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1089/2163] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:26.889214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187f27f-1603-466d-8762-f90095e9cd92 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:26.889709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:26.891016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.905737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:26.906539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.906714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5e5d-0f45-4c3e-9851-3112321a2dc2 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:26.907021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.907399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.910139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.912946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cfe26-4179-4d82-97ea-164e797d56a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:26.913680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cfe26-4179-4d82-97ea-164e797d56a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.914008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cfe26-4179-4d82-97ea-164e797d56a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.914292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cfe26-4179-4d82-97ea-164e797d56a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.914964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cfe26-4179-4d82-97ea-164e797d56a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:26.917514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cfe26-4179-4d82-97ea-164e797d56a9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:26.926878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:26.928698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:26.930164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1090/2164] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:26.931569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.931861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.932143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.944081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1075/2165] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:26.955671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:26.961261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:26.961418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:26.977202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1091/2166] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:26.977430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1e7c3-d559-4f8f-8e42-c355efe62f40 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:26.977617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:26.981572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:26.982507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.982866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.983205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.985612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6e75-7f47-4850-a3f2-86d7be7b60c9 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:26.991579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:26.992953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.992953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.992953 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.993417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:26.995341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:27.001746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:27.006660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:27.009121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1092/2167] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:27.009876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.012479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:27.022628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1076/2168] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:27.024194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.036079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03fde4-7114-4ad7-8cca-8d22210c41c6 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:27.039177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee8a9-c7ee-49ee-b29f-daa9414c4451 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:27.045973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db367b-ce09-4886-8670-77836ce4645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:27.046730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:27.046986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db367b-ce09-4886-8670-77836ce4645a service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:27.047349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db367b-ce09-4886-8670-77836ce4645a service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:27.047643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:27.047769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db367b-ce09-4886-8670-77836ce4645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:27.048047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:27.048391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] PATH_INFO: `/v3/projects/a8a050b5915f4317885c50cf9526d254/users/d67b90dc920c464a9859af39ebc3cc7b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:27.061717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db367b-ce09-4886-8670-77836ce4645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:27.063560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db367b-ce09-4886-8670-77836ce4645a service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:27.072144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a050b5915f4317885c50cf9526d254, user_id=d67b90dc920c464a9859af39ebc3cc7b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:27.076324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1093/2169] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:27.076958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:27.205957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1077/2170] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:27 2022] PUT /identity/v3/projects/a8a050b5915f4317885c50cf9526d254/users/d67b90dc920c464a9859af39ebc3cc7b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:27.220067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:27.221064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:27.221259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:27.263099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:27.296208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] MFA Rules not processed for user `d67b90dc920c464a9859af39ebc3cc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:27.298310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:27.400379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:27.457805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1094/2171] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:27.467565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.469950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:27.578484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae3ac-d607-4625-9999-bff0cf997089 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:27.584667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:27.585475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:27.586139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:27.586437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:27.598653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:27.600371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:27.616688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1078/2172] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:27.930816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.934454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.936124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:27.937543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:28.109087 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1942-8086-4f7a-99bb-80b3494983a7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:28.116855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:28.117924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.119506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.120162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] PATH_INFO: `/v3/projects/28b80cc1383e4c28bcc7a3699b9e2ad9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.121820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b80cc1383e4c28bcc7a3699b9e2ad9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:28.123998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:28.139513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b07b-79ac-4e87-a611-214b6c680dee service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:28.148069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:28.149203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.149661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.150134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] PATH_INFO: `/v3/projects/b8fe9373326148b4948ef6927950659f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.151724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fe9373326148b4948ef6927950659f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:28.154112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:28.209670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:28.298140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b80cc1383e4c28bcc7a3699b9e2ad9', 'request_id': 'req-14076786-6a61-4582-be57-7438ab3c6cac'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.302312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b80cc1383e4c28bcc7a3699b9e2ad9', 'request_id': 'req-14076786-6a61-4582-be57-7438ab3c6cac'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.350853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fe9373326148b4948ef6927950659f', 'request_id': 'req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.361693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fe9373326148b4948ef6927950659f', 'request_id': 'req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.370160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fe9373326148b4948ef6927950659f', 'request_id': 'req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.480085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b80cc1383e4c28bcc7a3699b9e2ad9', 'request_id': 'req-14076786-6a61-4582-be57-7438ab3c6cac'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.526121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] The token cache is being invalidate because project 28b80cc1383e4c28bcc7a3699b9e2ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:28.527754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.529444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.534980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1095/2173] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:27 2022] DELETE /identity/v3/projects/28b80cc1383e4c28bcc7a3699b9e2ad9 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:28.537212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:28.540140 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:28.669299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] The token cache is being invalidate because project b8fe9373326148b4948ef6927950659f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:28.669808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.676400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ca0f7f3-347b-4ed4-b52e-16b86dbbbb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.676400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1079/2174] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:27 2022] DELETE /identity/v3/projects/b8fe9373326148b4948ef6927950659f => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:28.677217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b97b0d-8a66-4ac3-b156-b97480c576cc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.677661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b97b0d-8a66-4ac3-b156-b97480c576cc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.678670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b97b0d-8a66-4ac3-b156-b97480c576cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.688420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076786-6a61-4582-be57-7438ab3c6cac admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:28.707059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-19b97b0d-8a66-4ac3-b156-b97480c576cc None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:28.708396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-19b97b0d-8a66-4ac3-b156-b97480c576cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:28.717172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:28.819524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1080/2175] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:28.821868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895d9cf-700d-4336-9935-64f05b7bfc2c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.833502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895d9cf-700d-4336-9935-64f05b7bfc2c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.833502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895d9cf-700d-4336-9935-64f05b7bfc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.851941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7895d9cf-700d-4336-9935-64f05b7bfc2c None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:28.853245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7895d9cf-700d-4336-9935-64f05b7bfc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:28.877960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1081/2176] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:28.885118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392e04a-d04a-4108-a775-dee8e158bcbb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.886542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392e04a-d04a-4108-a775-dee8e158bcbb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.887273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392e04a-d04a-4108-a775-dee8e158bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.937645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c392e04a-d04a-4108-a775-dee8e158bcbb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:28.941017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c392e04a-d04a-4108-a775-dee8e158bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:28.949597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:28.952569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.954157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.957717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] PATH_INFO: `/v3/projects/48a6cd393fcf4c68af3db7d4c457d6c2` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.961539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a6cd393fcf4c68af3db7d4c457d6c2)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:28.963379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1082/2177] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:28.966510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.970371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:28.971151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.971151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.006363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:29.008951 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:29.031605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1083/2178] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:29.031605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.059512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0318f7-c88a-4633-bb19-e39babb4967d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.071134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.072848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.073343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.073869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.075552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:29.076081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.079559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.110353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1084/2179] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:29.111286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.143862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7fd44-17ff-43ec-8fff-2ee0c4cb5876 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.150758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.151407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.151693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.151973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.152658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:29.152978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.154748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.179979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1085/2180] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:29.181067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.193543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09af1b-8199-4c6e-88bb-7b114623fb8b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.218115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.219062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.219357 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.219609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.220567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.222188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a6cd393fcf4c68af3db7d4c457d6c2', 'request_id': 'req-e6d993a8-71b5-4266-b6dc-600793188ab0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.222531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.230094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a6cd393fcf4c68af3db7d4c457d6c2', 'request_id': 'req-e6d993a8-71b5-4266-b6dc-600793188ab0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.372538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a6cd393fcf4c68af3db7d4c457d6c2', 'request_id': 'req-e6d993a8-71b5-4266-b6dc-600793188ab0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.396424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1086/2181] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:29.396424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.406565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] The token cache is being invalidate because project 48a6cd393fcf4c68af3db7d4c457d6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:29.410010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.413666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.420325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1096/2182] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:28 2022] DELETE /identity/v3/projects/48a6cd393fcf4c68af3db7d4c457d6c2 => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:29.422053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.429177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:29.736768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d993a8-71b5-4266-b6dc-600793188ab0 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.750541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.752629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.753250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.753813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.755017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.757568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59473a8-adc7-4f82-a1f6-63ef9819b565 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.759631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.771927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.776136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.776851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.777343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.778815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.782749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.994096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1097/2183] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/users => generated 291 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:29.995176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.000788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:30.032191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717718c-dc2e-4187-9908-6a9d47caad1a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.032627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1087/2184] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/projects => generated 371 bytes in 639 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:30.036505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.124633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.125302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.125605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.126011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] PATH_INFO: `/v3/users/0afc82f863bb46e0913f7390d56db473` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.128267 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afc82f863bb46e0913f7390d56db473)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.130780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.295271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e8176-f801-418d-9f79-83ffc7762a40 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.314840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.316073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.316679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.317276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.319243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.322552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.339265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1088/2185] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:30.339265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.341441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:30.388121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425a077-40e4-4c42-a1da-be90ba7e01e0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.397153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.398269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.399043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.399339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.400366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.403921 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.531253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afc82f863bb46e0913f7390d56db473'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:30.624313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1089/2186] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:30 2022] POST /identity/v3/users => generated 286 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:30.625504 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.635011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afc82f863bb46e0913f7390d56db473'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:30.638747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133c443-c9b1-4662-8d7e-246242e55af2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.652912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.653578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.653848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.654168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.654886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.656669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.657280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afc82f863bb46e0913f7390d56db473'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:30.658196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afc82f863bb46e0913f7390d56db473'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:30.669954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afc82f863bb46e0913f7390d56db473'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:30.670299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1090/2187] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:30.671783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.680684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:30.683305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1098/2188] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:29 2022] DELETE /identity/v3/users/0afc82f863bb46e0913f7390d56db473 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:30.685034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.687020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:30.719210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ea6d7-2340-493c-8ac5-d75a42dce471 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.904733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5f857-1e1b-4d9b-8e42-3b5a1fdbb644 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.922204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90796ec3-e47f-410c-a6f6-fda29ed43f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.927646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90796ec3-e47f-410c-a6f6-fda29ed43f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.928675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90796ec3-e47f-410c-a6f6-fda29ed43f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.928973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90796ec3-e47f-410c-a6f6-fda29ed43f2e admin admin] PATH_INFO: `/v3/projects/2e6d998735c243c3bcd08ca2e2d81ae9/users/e53d275b44a2432a9fe8611304bcfa91/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.937655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.941543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.941848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.943365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.945109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.951434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.977113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90796ec3-e47f-410c-a6f6-fda29ed43f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6d998735c243c3bcd08ca2e2d81ae9, user_id=e53d275b44a2432a9fe8611304bcfa91, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.984673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1091/2189] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:30.986215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90796ec3-e47f-410c-a6f6-fda29ed43f2e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.991511 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:31.007969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585b48e-97a2-4c41-a33d-1f6880ea0d4e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:31.016499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:31.017281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:31.017572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:31.025860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:31.026874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:31.028621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:31.051510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1092/2190] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:31.056606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:31.075455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476b464-b779-49be-8e2b-9b1d64cd6599 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:31.081635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da76545-533c-4d16-8b5a-655478c01510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:31.082536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da76545-533c-4d16-8b5a-655478c01510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:31.082914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da76545-533c-4d16-8b5a-655478c01510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:31.083300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da76545-533c-4d16-8b5a-655478c01510 admin admin] PATH_INFO: `/v3/projects/c1c061b824cb4a92a469061e7a67f967/users/af1f81b83784472abc4b6ff4393afb7c/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:31.105017 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da76545-533c-4d16-8b5a-655478c01510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c061b824cb4a92a469061e7a67f967, user_id=af1f81b83784472abc4b6ff4393afb7c, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:31.108644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da76545-533c-4d16-8b5a-655478c01510 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:31.120857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1099/2191] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:30 2022] PUT /identity/v3/projects/2e6d998735c243c3bcd08ca2e2d81ae9/users/e53d275b44a2432a9fe8611304bcfa91/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:31.127820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:31.128685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:31.129171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:31.162388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:31.163944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:31.286085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1100/2192] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:31.321596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:31.321596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:31.321883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1093/2193] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:31 2022] PUT /identity/v3/projects/c1c061b824cb4a92a469061e7a67f967/users/af1f81b83784472abc4b6ff4393afb7c/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:31.324793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:31.325295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:31.325788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:31.395866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:31.398192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:31.433729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829ca07-d216-4e8f-ad4c-09e830d05189 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:31.602932 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f0388-fc43-450e-a841-7fb985779c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:31.604793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f0388-fc43-450e-a841-7fb985779c5d service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:31.605313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f0388-fc43-450e-a841-7fb985779c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:31.605783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f0388-fc43-450e-a841-7fb985779c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:31.654563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1094/2194] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:31.668678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:31.709994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd0db9-0c1d-4479-a2bc-02a9714ea592 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:31.718183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:31.719881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:31.737235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:31.737235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:31.758982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:31.760580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:31.767187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f0388-fc43-450e-a841-7fb985779c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:31.770205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f0388-fc43-450e-a841-7fb985779c5d service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:31.774327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1095/2195] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:31.786564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1101/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:32.099563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:32.526272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:32.528132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:32.544170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c75cd6-e082-463c-a12f-8e79bfe16384 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:32.550925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:32.551726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:32.552117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:32.552597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] PATH_INFO: `/v3/projects/8fea2773d99e4d0c80df710daeaabbed` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:32.554334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fea2773d99e4d0c80df710daeaabbed)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:32.556479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:32.837686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fea2773d99e4d0c80df710daeaabbed', 'request_id': 'req-315f6cda-9e62-461e-a9cd-d7a9e46692f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.852761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fea2773d99e4d0c80df710daeaabbed', 'request_id': 'req-315f6cda-9e62-461e-a9cd-d7a9e46692f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.863765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fea2773d99e4d0c80df710daeaabbed', 'request_id': 'req-315f6cda-9e62-461e-a9cd-d7a9e46692f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.049306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.049992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.051138 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.059567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] The token cache is being invalidate because project 8fea2773d99e4d0c80df710daeaabbed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:33.063988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.063988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-315f6cda-9e62-461e-a9cd-d7a9e46692f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.064932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1096/2197] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:32 2022] DELETE /identity/v3/projects/8fea2773d99e4d0c80df710daeaabbed => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:33.071609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26670-c3cf-49a6-bc5a-bb6280545a9f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.072232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26670-c3cf-49a6-bc5a-bb6280545a9f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.072764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26670-c3cf-49a6-bc5a-bb6280545a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.105366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b0a26670-c3cf-49a6-bc5a-bb6280545a9f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:33.108007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b0a26670-c3cf-49a6-bc5a-bb6280545a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:33.139206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:33.190752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] MFA Rules not processed for user `7562a0429fdf42948174b517b211884f`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:33.192920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:33.258514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1097/2198] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:33.270130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.270720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.271162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.309867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:33.313829 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:33.338572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1098/2199] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:33.340585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:33.353354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1102/2200] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:33.360675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dffd7a-cb3e-478b-aaf8-33671197e454 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:33.371482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:33.373036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.373240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.374631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.375465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:33.376471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:33.379739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:33.399998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1099/2201] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:33.404532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:33.420175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92109309-736b-4da2-b1c3-cf2c996b03b6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:33.434638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:33.435288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.438794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.438794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.438794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:33.438794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:33.617119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:33.618261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1103/2202] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:33 2022] POST /identity/v3/projects => generated 409 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:33.639508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576b639-2ad3-4309-afd2-e1b5375eac93 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:33.772113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:33.774220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.774772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.775277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.776106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:33.779468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:34.071187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1100/2203] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:33 2022] POST /identity/v3/users => generated 305 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:34.076001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:34.091090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6428-42c9-46aa-ba35-c9c00cec802f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:34.099617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:34.100647 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:34.101200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:34.101633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:34.102561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:34.104539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:34.122565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1104/2204] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:34.127868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:34.143841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47960-5158-4ef3-89e7-a7a93e1b04ff admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:34.150506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfdc7-e7df-48a2-bde5-b32ecaf3bd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:34.151512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfdc7-e7df-48a2-bde5-b32ecaf3bd86 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:34.152009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfdc7-e7df-48a2-bde5-b32ecaf3bd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:34.152505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfdc7-e7df-48a2-bde5-b32ecaf3bd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:34.153344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfdc7-e7df-48a2-bde5-b32ecaf3bd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:34.155964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfdc7-e7df-48a2-bde5-b32ecaf3bd86 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:34.172952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1101/2205] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:34.179901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:34.195176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ed17e-9e61-4765-adb0-0b56d23309a3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:34.207284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:34.215207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:34.215207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:34.215207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] PATH_INFO: `/v3/projects/f30d79c0d7214bb2b324725bb6b3baee/users/bd59fdba759340d79d7aa04e4017a1ed/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:34.231277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30d79c0d7214bb2b324725bb6b3baee, user_id=bd59fdba759340d79d7aa04e4017a1ed, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:34.234956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:34.356948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1105/2206] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:34 2022] PUT /identity/v3/projects/f30d79c0d7214bb2b324725bb6b3baee/users/bd59fdba759340d79d7aa04e4017a1ed/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:34.369365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:34.370352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:34.370861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:34.441265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] MFA Rules not processed for user `bd59fdba759340d79d7aa04e4017a1ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:34.442585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:34.548394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1102/2207] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:34.550015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:34.673820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a6a27-14f5-44eb-860b-e4bce1b45add admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:34.683539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:34.684979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:34.686528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:34.686973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:34.687838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:34.694494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:34.828809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1106/2208] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:34 2022] POST /identity/v3/projects => generated 407 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:34.835955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:34.862100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455a493-ff69-4f18-9493-e3974b090aba None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.099419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.105741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.106557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.107330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.113666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.116434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.294410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1103/2209] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:34 2022] POST /identity/v3/users => generated 310 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:35.298569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.382460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958de6-f603-4a1e-9859-53772f9af062 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.395380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.397566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.398914 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.399296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.400072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.403125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.421476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1107/2210] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:35.427208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.462845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036eec3-585a-4fcc-8b72-153269961746 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.472470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608fa8d-0827-433f-9f53-e828ba4564bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.474053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608fa8d-0827-433f-9f53-e828ba4564bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.474383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608fa8d-0827-433f-9f53-e828ba4564bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.474895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608fa8d-0827-433f-9f53-e828ba4564bc admin admin] PATH_INFO: `/v3/projects/7ce54a4a8da74dac87b23e7bc8ab11ae/users/029cd17c90394694ab201c01ecdd4c9e/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.512853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608fa8d-0827-433f-9f53-e828ba4564bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce54a4a8da74dac87b23e7bc8ab11ae, user_id=029cd17c90394694ab201c01ecdd4c9e, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.518987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608fa8d-0827-433f-9f53-e828ba4564bc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.729085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1104/2211] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:35 2022] PUT /identity/v3/projects/7ce54a4a8da74dac87b23e7bc8ab11ae/users/029cd17c90394694ab201c01ecdd4c9e/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:35.732015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.762749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818cc84-5eb2-4ba2-9010-46ca68c10fd1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.832454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.832779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.833165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.906464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] MFA Rules not processed for user `e53d275b44a2432a9fe8611304bcfa91`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:35.909540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:35.947539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.948375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.948673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.949053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.949625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.952149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.985999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1108/2212] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:35.988457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.020957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f54f2-a3a0-4dab-b6ec-a52336b8c5f4 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.030732 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59327c-c702-4d86-9e0a-3034b7dff259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.031442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59327c-c702-4d86-9e0a-3034b7dff259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.031728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59327c-c702-4d86-9e0a-3034b7dff259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.032007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59327c-c702-4d86-9e0a-3034b7dff259 admin admin] PATH_INFO: `/v3/projects/7ce54a4a8da74dac87b23e7bc8ab11ae/users/029cd17c90394694ab201c01ecdd4c9e/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.037221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:36.039968 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59327c-c702-4d86-9e0a-3034b7dff259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce54a4a8da74dac87b23e7bc8ab11ae, user_id=029cd17c90394694ab201c01ecdd4c9e, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.048772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59327c-c702-4d86-9e0a-3034b7dff259 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.097861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1109/2213] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:35 2022] PUT /identity/v3/projects/7ce54a4a8da74dac87b23e7bc8ab11ae/users/029cd17c90394694ab201c01ecdd4c9e/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:36.110206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.110484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.110628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.119898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1105/2214] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:36.144701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.147833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:36.200661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:36.250334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] MFA Rules not processed for user `029cd17c90394694ab201c01ecdd4c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:36.258717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:36.414740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1535e08-ede3-4494-bda7-5831c5bbea76 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.440586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1110/2215] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:36.453660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.453660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:36.544503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f20944-193a-447f-aa1c-f843c25ee5d6 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.555566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.557611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.558319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.558949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.589435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.591533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.592282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.593369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.601457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.602165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.619580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1111/2216] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:36.756978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.760555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.802151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1106/2217] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:37.401399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:37.416244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18592e-65e6-4a29-9130-23b2d6907c35 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:37.427376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2c4bd-b0f8-4d02-bc8b-bb1566c9ee03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:37.428023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2c4bd-b0f8-4d02-bc8b-bb1566c9ee03 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:37.428307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2c4bd-b0f8-4d02-bc8b-bb1566c9ee03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:37.428589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2c4bd-b0f8-4d02-bc8b-bb1566c9ee03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:37.533449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2c4bd-b0f8-4d02-bc8b-bb1566c9ee03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:37.535352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2c4bd-b0f8-4d02-bc8b-bb1566c9ee03 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:37.562338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1112/2218] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:39.068798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:40.134514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.136174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.217609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:40.321479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced494b2-7b74-4dfa-a7af-4e2f41a5960f service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.333243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.334288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.338952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.341399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.345342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.362311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.364847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2ac8c-43c3-42ae-b5bb-008e2b05d249 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.379719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1107/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:40.706163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.707994 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.708785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.805054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] MFA Rules not processed for user `e53d275b44a2432a9fe8611304bcfa91`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:40.806381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.828218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1113/2220] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:44.041300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:44.710503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:45.742409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:45.743305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:45.743305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:45.784195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] MFA Rules not processed for user `af1f81b83784472abc4b6ff4393afb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:45.786424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:45.903226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1108/2221] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:45.917363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:45.917363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:46.041926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bbe49-ff41-483f-a248-976c3bb5d7f5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.052041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.053319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.053885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.054489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.055806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.060165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:46.263730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1114/2222] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:45 2022] POST /identity/v3/projects => generated 371 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:46.267427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.283465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2902fb-af1a-430b-ac93-4cfbec47b1b7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.410738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.411741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.412121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.412382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.413533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.417282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:46.613042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1109/2223] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:46 2022] POST /identity/v3/users => generated 292 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:46.618303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.635331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3b63c-2ed3-42f2-a664-6ac59e4f7f06 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.647843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.649359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.650758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.653062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.654121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.657348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:46.678091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1115/2224] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:46.678786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.721118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcedb6-bb60-40a9-becd-5fdd27279883 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.729025 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.730412 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.737689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.737689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] PATH_INFO: `/v3/projects/f394f5c330444b81b46cb0ba3f026cf0/users/ac5265f19dc9475e889a5816c97ef624/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.776656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f394f5c330444b81b46cb0ba3f026cf0, user_id=ac5265f19dc9475e889a5816c97ef624, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.792113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:46.805263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:46.956899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1110/2225] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:46 2022] PUT /identity/v3/projects/f394f5c330444b81b46cb0ba3f026cf0/users/ac5265f19dc9475e889a5816c97ef624/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:46.964633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.974733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:47.005524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b66bfe-0fe5-4771-8366-bf72fd05a61c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:47.138883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:47.139551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.140033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.140319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.141014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:47.143578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.156321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1116/2226] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:47.162513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:47.181521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de826a8-aef5-40c0-a12c-db469549e29f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:47.189873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3fdfc-11c1-4c08-9750-d677ea365c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:47.191104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3fdfc-11c1-4c08-9750-d677ea365c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.191729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3fdfc-11c1-4c08-9750-d677ea365c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.192130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3fdfc-11c1-4c08-9750-d677ea365c79 admin admin] PATH_INFO: `/v3/projects/f394f5c330444b81b46cb0ba3f026cf0/users/ac5265f19dc9475e889a5816c97ef624/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.195819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3fdfc-11c1-4c08-9750-d677ea365c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f394f5c330444b81b46cb0ba3f026cf0, user_id=ac5265f19dc9475e889a5816c97ef624, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:47.201313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3fdfc-11c1-4c08-9750-d677ea365c79 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.223172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1111/2227] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:47 2022] PUT /identity/v3/projects/f394f5c330444b81b46cb0ba3f026cf0/users/ac5265f19dc9475e889a5816c97ef624/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:50.103676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:50.860634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:51.844928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:52.469491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:54.034713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:54.037239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:54.060788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879e7c1-934d-410e-8fbd-770f4859b016 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:54.190464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:54.190464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:54.190464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:54.190464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] PATH_INFO: `/v3/users/d67b90dc920c464a9859af39ebc3cc7b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:54.191022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67b90dc920c464a9859af39ebc3cc7b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:54.191022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:54.531042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67b90dc920c464a9859af39ebc3cc7b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.680620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67b90dc920c464a9859af39ebc3cc7b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.703562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67b90dc920c464a9859af39ebc3cc7b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.707331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67b90dc920c464a9859af39ebc3cc7b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.716135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67b90dc920c464a9859af39ebc3cc7b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.725296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1117/2228] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:54 2022] DELETE /identity/v3/users/d67b90dc920c464a9859af39ebc3cc7b => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:54.730825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:54.730825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:54.731480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:54.755260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:54.756538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:54.861302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1112/2229] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:54.874290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:54.878718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:54.969211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f391711-465e-4675-b7d3-0a42e36d0fe5 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:54.977573 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1a9d7-bcf3-4b56-a499-64c2972f4d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:54.978575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1a9d7-bcf3-4b56-a499-64c2972f4d04 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:54.979039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1a9d7-bcf3-4b56-a499-64c2972f4d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:54.979493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1a9d7-bcf3-4b56-a499-64c2972f4d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:54.999012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1a9d7-bcf3-4b56-a499-64c2972f4d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:55.000610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1a9d7-bcf3-4b56-a499-64c2972f4d04 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:55.035407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1118/2230] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:55.058642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:55.352282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:55.355266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:55.385933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3715b21-fdc2-4c6c-9e3e-c29ef84e0fc1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:55.394902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:55.396019 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:55.396759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:55.397468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] PATH_INFO: `/v3/users/bd59fdba759340d79d7aa04e4017a1ed` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:55.400000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd59fdba759340d79d7aa04e4017a1ed)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:55.404944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:55.708172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd59fdba759340d79d7aa04e4017a1ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.712241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd59fdba759340d79d7aa04e4017a1ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.734610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd59fdba759340d79d7aa04e4017a1ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.904300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:55.905052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd59fdba759340d79d7aa04e4017a1ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.917919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd59fdba759340d79d7aa04e4017a1ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.935082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1113/2231] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:55 2022] DELETE /identity/v3/users/bd59fdba759340d79d7aa04e4017a1ed => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:55.941225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:55.941747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:55.942092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:55.995581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:55.998092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:56.110846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1119/2232] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:56.120356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:56.122552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:56.187729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:56.190361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:56.207315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bab9f-506c-43b3-bcfa-d9d2d9c4ef20 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:56.219692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:56.241160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.241160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.241160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] PATH_INFO: `/v3/projects/a8a050b5915f4317885c50cf9526d254` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.241160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a050b5915f4317885c50cf9526d254)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:56.241160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.264408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7be108-94dd-47f3-82f2-9c60ee265d84 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:56.273711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:56.275170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.275836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.276543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.296550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:56.298350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.311464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1114/2233] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:56.422420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a050b5915f4317885c50cf9526d254', 'request_id': 'req-09eaaff0-831d-4355-9ef9-f0d6f8263dde'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.434394 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a050b5915f4317885c50cf9526d254', 'request_id': 'req-09eaaff0-831d-4355-9ef9-f0d6f8263dde'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.589405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a050b5915f4317885c50cf9526d254', 'request_id': 'req-09eaaff0-831d-4355-9ef9-f0d6f8263dde'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.619910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] The token cache is being invalidate because project a8a050b5915f4317885c50cf9526d254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:56.628913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.628913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09eaaff0-831d-4355-9ef9-f0d6f8263dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.629685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1120/2234] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:56 2022] DELETE /identity/v3/projects/a8a050b5915f4317885c50cf9526d254 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:57.275132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:57.449276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.452849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:57.479136 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cff19-e973-4eea-b76e-b714b78a3cbd service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.637297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.638425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.639837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.640335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] PATH_INFO: `/v3/projects/f30d79c0d7214bb2b324725bb6b3baee` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.643147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30d79c0d7214bb2b324725bb6b3baee)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.645840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.826712 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30d79c0d7214bb2b324725bb6b3baee', 'request_id': 'req-9be260d2-201c-461c-a384-3a4ba2c67101'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.839194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e738d-2f34-4f8b-904b-862ce3cce0e0 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.839537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e738d-2f34-4f8b-904b-862ce3cce0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.839808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e738d-2f34-4f8b-904b-862ce3cce0e0 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.841676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1121/2235] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:08:57.851520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.851998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.852431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.854711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30d79c0d7214bb2b324725bb6b3baee', 'request_id': 'req-9be260d2-201c-461c-a384-3a4ba2c67101'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.886007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30d79c0d7214bb2b324725bb6b3baee', 'request_id': 'req-9be260d2-201c-461c-a384-3a4ba2c67101'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.897393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:57.898758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.020317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:58.038881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] The token cache is being invalidate because project f30d79c0d7214bb2b324725bb6b3baee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:58.039314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.040640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.042854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1115/2236] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:57 2022] DELETE /identity/v3/projects/f30d79c0d7214bb2b324725bb6b3baee => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:58.047226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.049803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.103999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1122/2237] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:58.126676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.131413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.248174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be260d2-201c-461c-a384-3a4ba2c67101 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.255306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d551-80ab-4496-a75b-8a59ead0134d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.263749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.272459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.272459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.272459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] PATH_INFO: `/v3/users/029cd17c90394694ab201c01ecdd4c9e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.273216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced6509-9897-4152-ab6b-1282e461c74a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.273216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced6509-9897-4152-ab6b-1282e461c74a service glance] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.273216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced6509-9897-4152-ab6b-1282e461c74a service glance] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.273216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced6509-9897-4152-ab6b-1282e461c74a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.273783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] RBAC: Authorizing `identity:delete_user(user_id=029cd17c90394694ab201c01ecdd4c9e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.273783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.298760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced6509-9897-4152-ab6b-1282e461c74a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.305798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced6509-9897-4152-ab6b-1282e461c74a service glance] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.381395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1123/2238] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:58.810018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029cd17c90394694ab201c01ecdd4c9e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.810832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029cd17c90394694ab201c01ecdd4c9e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.825559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029cd17c90394694ab201c01ecdd4c9e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.891950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e008-9867-425a-aa75-31f1c0a66058 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.892444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e008-9867-425a-aa75-31f1c0a66058 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.892958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e008-9867-425a-aa75-31f1c0a66058 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.898844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1124/2239] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:08:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:08:58.906289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b728437-14e8-44b9-aa1e-189eadb0e4f3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.906884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b728437-14e8-44b9-aa1e-189eadb0e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.907927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b728437-14e8-44b9-aa1e-189eadb0e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.909710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029cd17c90394694ab201c01ecdd4c9e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.921032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029cd17c90394694ab201c01ecdd4c9e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.943223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1116/2240] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:58 2022] DELETE /identity/v3/users/029cd17c90394694ab201c01ecdd4c9e => generated 0 bytes in 892 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:58.961284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8b728437-14e8-44b9-aa1e-189eadb0e4f3 None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:58.964153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8b728437-14e8-44b9-aa1e-189eadb0e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.089929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1125/2241] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:59.121749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:59.123635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.154111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbb999-8d16-437f-a0e8-adcc1ce7027a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:59.162470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c964d-4ed8-4269-b823-226188919fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:59.165331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c964d-4ed8-4269-b823-226188919fbe service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.170917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c964d-4ed8-4269-b823-226188919fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.171608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c964d-4ed8-4269-b823-226188919fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.195472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c964d-4ed8-4269-b823-226188919fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:59.204651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c964d-4ed8-4269-b823-226188919fbe service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:59.228249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1117/2242] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:59.524643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1f151-bade-4dc2-aba0-9e909933bd79 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.525246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1f151-bade-4dc2-aba0-9e909933bd79 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.525648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1f151-bade-4dc2-aba0-9e909933bd79 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.528800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1126/2243] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:08:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:08:59.540727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.541231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.541735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.586549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:59.588747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.706545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1118/2244] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:59.746656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1f151-bade-4dc2-aba0-9e909933bd79 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:59.749988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-92a1f151-bade-4dc2-aba0-9e909933bd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.776216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1f151-bade-4dc2-aba0-9e909933bd79 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:59.790929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eec217-2e0d-489b-b41e-61f3247c43a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:59.791576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eec217-2e0d-489b-b41e-61f3247c43a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.794371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eec217-2e0d-489b-b41e-61f3247c43a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.794371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eec217-2e0d-489b-b41e-61f3247c43a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.816273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eec217-2e0d-489b-b41e-61f3247c43a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:59.817769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eec217-2e0d-489b-b41e-61f3247c43a8 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:59.832024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1127/2245] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:00.082519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.086352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.349462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd501ce-fca7-420f-abad-17d247445395 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.366032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.367433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.368010 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.368648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] PATH_INFO: `/v3/projects/7ce54a4a8da74dac87b23e7bc8ab11ae` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.370633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce54a4a8da74dac87b23e7bc8ab11ae)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.373186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.528762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.529308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.529481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.575710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] MFA Rules not processed for user `ac5265f19dc9475e889a5816c97ef624`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:00.578076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.658232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce54a4a8da74dac87b23e7bc8ab11ae', 'request_id': 'req-f6734a49-017c-40fe-8338-cc9770cd38b8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.668935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce54a4a8da74dac87b23e7bc8ab11ae', 'request_id': 'req-f6734a49-017c-40fe-8338-cc9770cd38b8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.677431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce54a4a8da74dac87b23e7bc8ab11ae', 'request_id': 'req-f6734a49-017c-40fe-8338-cc9770cd38b8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.713975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1128/2246] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:00.727505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.727505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.742202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91484b-c73f-4e86-bb71-8cb7593bd62c None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.799716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] The token cache is being invalidate because project 7ce54a4a8da74dac87b23e7bc8ab11ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:00.800016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.800838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f6734a49-017c-40fe-8338-cc9770cd38b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.806937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1119/2247] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:00 2022] DELETE /identity/v3/projects/7ce54a4a8da74dac87b23e7bc8ab11ae => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:00.808820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb651956-b6fb-488d-a7a8-ec111d0c5994 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.809113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb651956-b6fb-488d-a7a8-ec111d0c5994 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.809391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb651956-b6fb-488d-a7a8-ec111d0c5994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.838321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-eb651956-b6fb-488d-a7a8-ec111d0c5994 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:00.839738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eb651956-b6fb-488d-a7a8-ec111d0c5994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.871519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.872167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.872447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.872736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.875289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.913945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1120/2248] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:00.921714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.922121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.922462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.950852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:00.952155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.963378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1121/2249] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:00.969119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.971279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.973038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.980500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45d02-fd70-456e-88dd-4a885be1d37a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.985042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1129/2250] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:09:00.986142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.988450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.988450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.988450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.988880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:00.989183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.990889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.007097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1122/2251] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:01.013938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.032421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:01.051785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87183f94-d0fd-4235-bdc3-639c4485fa5b service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.063525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.064415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.064814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.065212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.066570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.069202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.344048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1130/2252] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:01 2022] POST /identity/v3/projects => generated 387 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:01.347087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.369249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4f848-a63c-4332-be26-1a39e8b957db admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.485935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.487280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.487572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.487919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.488620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.495844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.679759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1123/2253] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:01 2022] POST /identity/v3/users => generated 294 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:01.681854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.737284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57f51-11d0-4d56-be37-21763b4f4c02 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.764012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:01.791322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.792163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.792640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.792927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.793623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.795527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.828964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.832530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1131/2254] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:01.844602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bd741-bf3f-4824-836d-dc4d30d588ae admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.859822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920e303-74da-4406-9dea-de4a1cf16e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.860450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e303-74da-4406-9dea-de4a1cf16e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.860734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e303-74da-4406-9dea-de4a1cf16e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.861012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e303-74da-4406-9dea-de4a1cf16e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.862353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920e303-74da-4406-9dea-de4a1cf16e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.864251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920e303-74da-4406-9dea-de4a1cf16e7e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.891334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1124/2255] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:01.892819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.927088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe7a2e-6c8b-4428-b261-9d914eeaadb4 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.954298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.957684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.958222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.962308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] PATH_INFO: `/v3/projects/282c5fbf73f442daa7e300e34e00e460/users/6fb73884eee2482c8d4e309ee37549bb/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.010085 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282c5fbf73f442daa7e300e34e00e460, user_id=6fb73884eee2482c8d4e309ee37549bb, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:02.016503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:02.117009 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1132/2256] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:01 2022] PUT /identity/v3/projects/282c5fbf73f442daa7e300e34e00e460/users/6fb73884eee2482c8d4e309ee37549bb/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:02.138306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:02.138808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:02.139282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.190701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] MFA Rules not processed for user `6fb73884eee2482c8d4e309ee37549bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:02.194171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:02.342501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1125/2257] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:02.357006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:02.483386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28215f-df51-4411-80cf-7ced3040a150 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:02.516282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:02.516976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:02.517265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:02.517555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.518275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:02.520453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:02.707936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1133/2258] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:02 2022] POST /identity/v3/projects => generated 387 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:02.712568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:02.725654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbf67-6075-48a3-a13a-515aa1f08d01 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:02.836599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:02.837575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:02.838175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:02.838572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.839382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:02.842157 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:02.948312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1126/2259] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:02 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:02.952935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:02.977563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b2aa-2d2a-4db6-afe1-bebf3f1cd9f9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:02.988155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:02.989125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:02.989523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:02.989814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.990974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:02.996295 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:03.014816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1134/2260] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:03.021198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:03.035461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d7ce0-425c-419a-9825-e5541bb6f485 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:03.040920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:03.041620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:03.041977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:03.042264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] PATH_INFO: `/v3/projects/efabd92b93334fc9bd3fd19d2011b5a9/users/52392a370f794f169bf1ae88ba67d202/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:03.072731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efabd92b93334fc9bd3fd19d2011b5a9, user_id=52392a370f794f169bf1ae88ba67d202, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:03.080144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:03.274645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1127/2261] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:03 2022] PUT /identity/v3/projects/efabd92b93334fc9bd3fd19d2011b5a9/users/52392a370f794f169bf1ae88ba67d202/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:03.280754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:03.338265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46fd5a-8431-48d9-827c-a95bc9be0a34 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:03.495729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389416-5716-409b-8c24-121ff4215021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:03.496387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389416-5716-409b-8c24-121ff4215021 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:03.496671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389416-5716-409b-8c24-121ff4215021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:03.496954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389416-5716-409b-8c24-121ff4215021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:03.497620 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389416-5716-409b-8c24-121ff4215021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:03.501419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389416-5716-409b-8c24-121ff4215021 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:03.520176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1135/2262] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:03.529135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:03.551538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f24242-fe77-4c6a-8a9a-bdc8a8c8d063 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:03.561615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:03.563289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:03.564020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:03.565258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] PATH_INFO: `/v3/projects/efabd92b93334fc9bd3fd19d2011b5a9/users/52392a370f794f169bf1ae88ba67d202/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:03.568594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efabd92b93334fc9bd3fd19d2011b5a9, user_id=52392a370f794f169bf1ae88ba67d202, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:03.571983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:03.598007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1128/2263] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:03 2022] PUT /identity/v3/projects/efabd92b93334fc9bd3fd19d2011b5a9/users/52392a370f794f169bf1ae88ba67d202/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:03.600049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:03.612184 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:03.612743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:03.707210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] MFA Rules not processed for user `52392a370f794f169bf1ae88ba67d202`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:03.710904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:03.856804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1136/2264] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:03.872574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:03.876496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:04.019574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582795f-7e19-43fc-a4c7-7edc78b0e0e1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:04.063988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bba51b-eaaf-46b2-a435-129af234aba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:04.065431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bba51b-eaaf-46b2-a435-129af234aba6 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:04.066729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bba51b-eaaf-46b2-a435-129af234aba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:04.067535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bba51b-eaaf-46b2-a435-129af234aba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:04.095292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bba51b-eaaf-46b2-a435-129af234aba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:04.101171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bba51b-eaaf-46b2-a435-129af234aba6 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:04.133889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1129/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:04.636308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:05.350393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:06.071760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:06.209887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30144e-85d9-43f9-ac40-dbea1c1003b9 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:06.214800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:06.215455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:06.215967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:06.216240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] PATH_INFO: `/v3/users/6fb73884eee2482c8d4e309ee37549bb` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:06.218355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb73884eee2482c8d4e309ee37549bb)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:06.221689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:06.638989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107364-5ca0-4596-aff7-ad9031f09eca None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:06.640290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107364-5ca0-4596-aff7-ad9031f09eca None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:06.640607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107364-5ca0-4596-aff7-ad9031f09eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:06.669529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb73884eee2482c8d4e309ee37549bb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.679299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-85107364-5ca0-4596-aff7-ad9031f09eca None None] MFA Rules not processed for user `af1f81b83784472abc4b6ff4393afb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:06.681881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-85107364-5ca0-4596-aff7-ad9031f09eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:06.766822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1130/2266] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:06.780992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb73884eee2482c8d4e309ee37549bb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.813175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb73884eee2482c8d4e309ee37549bb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.814318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb73884eee2482c8d4e309ee37549bb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.833614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb73884eee2482c8d4e309ee37549bb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.853235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1137/2267] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:06 2022] DELETE /identity/v3/users/6fb73884eee2482c8d4e309ee37549bb => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:06.855550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:06.858581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:06.858581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:06.899586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:06.899586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:07.082164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1131/2268] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:07.136197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:07.139458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:07.277854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dab7c-1e25-465e-866d-b3875ffd77bb admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:07.297133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9ebae-b8bd-40a0-accc-de6b1bfd4b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:07.300165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9ebae-b8bd-40a0-accc-de6b1bfd4b5e service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:07.300860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9ebae-b8bd-40a0-accc-de6b1bfd4b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:07.301150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9ebae-b8bd-40a0-accc-de6b1bfd4b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:07.316856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9ebae-b8bd-40a0-accc-de6b1bfd4b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:07.317747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9ebae-b8bd-40a0-accc-de6b1bfd4b5e service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:07.332306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1138/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:07.935612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:08.022344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.023022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.036274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd9427-ebb5-4997-a5e2-8b8e2a59fcdb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.050994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.050994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.050994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.050994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] PATH_INFO: `/v3/projects/282c5fbf73f442daa7e300e34e00e460` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.051410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282c5fbf73f442daa7e300e34e00e460)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.051410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.182700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.183478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.183478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.208655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:08.214397 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.232034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282c5fbf73f442daa7e300e34e00e460', 'request_id': 'req-8746d626-8e42-489a-ba5b-ac8ecd739c75'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.244697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282c5fbf73f442daa7e300e34e00e460', 'request_id': 'req-8746d626-8e42-489a-ba5b-ac8ecd739c75'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.255805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282c5fbf73f442daa7e300e34e00e460', 'request_id': 'req-8746d626-8e42-489a-ba5b-ac8ecd739c75'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.314227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1139/2270] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:09:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:08.333501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.336156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.343840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] The token cache is being invalidate because project 282c5fbf73f442daa7e300e34e00e460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:08.344910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.347141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.354876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1132/2271] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:08 2022] DELETE /identity/v3/projects/282c5fbf73f442daa7e300e34e00e460 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:08.357103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.359679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.530377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff03aa7-2682-4b98-8d48-4997149a84fe None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.540428 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.542172 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.544524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.547064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.550980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.552989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746d626-8e42-489a-ba5b-ac8ecd739c75 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.566114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.567743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.568493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.569080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] PATH_INFO: `/v3/users/52392a370f794f169bf1ae88ba67d202` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.572760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52392a370f794f169bf1ae88ba67d202)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.578897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.609037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.613883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.654057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1140/2272] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:09:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:09:08.775561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:08.930498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52392a370f794f169bf1ae88ba67d202'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.931392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52392a370f794f169bf1ae88ba67d202'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.996375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52392a370f794f169bf1ae88ba67d202'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:09.137960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52392a370f794f169bf1ae88ba67d202'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:09.155527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01472377-70a6-4a30-bf29-1cc5da85a40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52392a370f794f169bf1ae88ba67d202'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:09.176142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1133/2273] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:08 2022] DELETE /identity/v3/users/52392a370f794f169bf1ae88ba67d202 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:09.754770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:10.872281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:10.912722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:10.938030 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15047b-afb5-4330-9c96-8e84397ef574 service swift] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:11.016923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:11.151045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:11.154574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:11.158413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:11.158901 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] PATH_INFO: `/v3/projects/efabd92b93334fc9bd3fd19d2011b5a9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:11.163410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efabd92b93334fc9bd3fd19d2011b5a9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:11.179110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:11.523938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efabd92b93334fc9bd3fd19d2011b5a9', 'request_id': 'req-b282e89b-d010-4b2f-872b-e91f5fca6d42'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:11.552868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efabd92b93334fc9bd3fd19d2011b5a9', 'request_id': 'req-b282e89b-d010-4b2f-872b-e91f5fca6d42'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:11.686319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efabd92b93334fc9bd3fd19d2011b5a9', 'request_id': 'req-b282e89b-d010-4b2f-872b-e91f5fca6d42'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:11.704560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] The token cache is being invalidate because project efabd92b93334fc9bd3fd19d2011b5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:11.704852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:11.706276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-b282e89b-d010-4b2f-872b-e91f5fca6d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:11.709002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1141/2274] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:10 2022] DELETE /identity/v3/projects/efabd92b93334fc9bd3fd19d2011b5a9 => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:11.718263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:11.718753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:11.719275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:11.820379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:11.822788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:11.977018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:12.131584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1134/2275] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:12.141084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:12.142046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:12.142046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:12.242418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:12.245112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:12.297807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1142/2276] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:12.310902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:12.333657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ea84-d59d-4ddd-bbbb-038308fa045b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:12.342682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:12.343366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:12.343680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:12.343990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:12.344769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:12.345146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:12.349130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:12.367499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1135/2277] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:12.379384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:12.393355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a37e9-ff62-4717-b26f-b3408e3b95f5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:12.401685 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:12.402950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:12.403830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:12.404361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:12.405727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:12.415132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:12.419632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:12.586320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1143/2278] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:12 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:12.590828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:12.608927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bce2c0-90cd-43c6-b95c-c1bbad6dcb7b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:12.781735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:12.784385 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:12.785179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:12.788077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:12.788077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:12.795487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:12.813683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:12.925572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1136/2279] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:12 2022] POST /identity/v3/users => generated 292 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:12.931807 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:12.954207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b7ac-fc3b-40a4-bdd2-5dd0f3007605 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:12.961357 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:12.962705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:12.963557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:12.964173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:12.965456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:12.967446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:13.014022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1144/2280] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:13.014658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:13.047560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8098c7b-1cfd-40ac-aac0-3c99e4bc881b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:13.063273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cd018-219c-475f-9389-1d1dcf4fb774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:13.065774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cd018-219c-475f-9389-1d1dcf4fb774 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:13.066925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cd018-219c-475f-9389-1d1dcf4fb774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:13.068366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cd018-219c-475f-9389-1d1dcf4fb774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:13.070187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cd018-219c-475f-9389-1d1dcf4fb774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:13.073569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cd018-219c-475f-9389-1d1dcf4fb774 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:13.099866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1137/2281] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:13.108000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:13.130589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b6e3-8860-4e12-a227-3697edb2c8d6 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:13.142625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:13.144782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:13.145326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:13.145884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] PATH_INFO: `/v3/projects/354868dd95a448c28ade48bc4e4c38a7/users/7402bfe6b7cc4791bb091ee8197ebbcf/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:13.193459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=354868dd95a448c28ade48bc4e4c38a7, user_id=7402bfe6b7cc4791bb091ee8197ebbcf, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:13.212013 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:13.369875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1145/2282] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:13 2022] PUT /identity/v3/projects/354868dd95a448c28ade48bc4e4c38a7/users/7402bfe6b7cc4791bb091ee8197ebbcf/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:13.497139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:13.497139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:13.497139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:13.608071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] MFA Rules not processed for user `7402bfe6b7cc4791bb091ee8197ebbcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:13.610398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:13.778857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1138/2283] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:13.815062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:13.818742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:14.060602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6d9e-4273-47d3-a199-c5bbbbcefdee admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:14.084244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd1c36-547f-4bda-a851-b38d3deb0b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:14.096185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd1c36-547f-4bda-a851-b38d3deb0b0c service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:14.096185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd1c36-547f-4bda-a851-b38d3deb0b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:14.096185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd1c36-547f-4bda-a851-b38d3deb0b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:14.142350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd1c36-547f-4bda-a851-b38d3deb0b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:14.145978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd1c36-547f-4bda-a851-b38d3deb0b0c service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:14.186245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1146/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:14.239901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:14.482155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09af11e-ed35-4c47-9c6a-df9aeae449ac None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:14.504941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:14.505718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:14.505883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:14.506249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] PATH_INFO: `/v3/users/7402bfe6b7cc4791bb091ee8197ebbcf` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:14.507456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7402bfe6b7cc4791bb091ee8197ebbcf)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:14.515102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:14.957554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7402bfe6b7cc4791bb091ee8197ebbcf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:14.965368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7402bfe6b7cc4791bb091ee8197ebbcf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.010674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7402bfe6b7cc4791bb091ee8197ebbcf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.132403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7402bfe6b7cc4791bb091ee8197ebbcf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.141639 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7402bfe6b7cc4791bb091ee8197ebbcf'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.151133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1139/2285] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:14 2022] DELETE /identity/v3/users/7402bfe6b7cc4791bb091ee8197ebbcf => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:15.157246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:15.158860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:15.158860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:15.194666 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:15.197608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:15.301597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:15.322316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1147/2286] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:15.336927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:15.338549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:15.346582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:15.431923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c425183-b715-4748-847c-bbd32bb4b901 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:15.437632 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50483514-96c4-43f7-ad80-95fc51bd4aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:15.438314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483514-96c4-43f7-ad80-95fc51bd4aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:15.438599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483514-96c4-43f7-ad80-95fc51bd4aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:15.438923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483514-96c4-43f7-ad80-95fc51bd4aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:15.455836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50483514-96c4-43f7-ad80-95fc51bd4aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:15.455836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50483514-96c4-43f7-ad80-95fc51bd4aa6 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:15.465487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1140/2287] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:16.227386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:16.230272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:16.264396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e328cc-dfa8-4cae-894c-59a743516f88 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:16.265588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:16.266822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.267170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.267735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] PATH_INFO: `/v3/projects/354868dd95a448c28ade48bc4e4c38a7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.268939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354868dd95a448c28ade48bc4e4c38a7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:16.271026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:16.446710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354868dd95a448c28ade48bc4e4c38a7', 'request_id': 'req-a991a13d-624d-4e49-be9f-b30f7ea261a9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.454051 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354868dd95a448c28ade48bc4e4c38a7', 'request_id': 'req-a991a13d-624d-4e49-be9f-b30f7ea261a9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.561406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354868dd95a448c28ade48bc4e4c38a7', 'request_id': 'req-a991a13d-624d-4e49-be9f-b30f7ea261a9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.575757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] The token cache is being invalidate because project 354868dd95a448c28ade48bc4e4c38a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:16.582335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.582335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a991a13d-624d-4e49-be9f-b30f7ea261a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.582335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1148/2288] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:16 2022] DELETE /identity/v3/projects/354868dd95a448c28ade48bc4e4c38a7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:16.585764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.586583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.587576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.649743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:16.652732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:16.659821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:16.787201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1141/2289] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:16.789187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.789702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.790167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.816605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:16.817056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:16.826342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1149/2290] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:16.831672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:16.843986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abb4cd-4276-420d-826e-a3366f32b5e7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:16.851383 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:16.852183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.852485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.852802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.853450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:16.853761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:16.855594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:16.880190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1142/2291] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:09:16.886254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:16.904732 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2778ba-4312-414a-9cd0-250d5eaf4c16 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:16.916079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:16.919802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.919802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.919802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.919802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:16.925195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.051987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1150/2292] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:16 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:17.056885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.073050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a97674-2f8a-45ab-b98b-d14292c270c3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:17.253516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:17.254381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.254742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.255033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.255668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:17.277706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.437702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1143/2293] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:17 2022] POST /identity/v3/users => generated 292 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:17.446312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.465076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebf492-7b78-4713-923e-e8bfd6b8c8ce admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:17.479161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:17.479792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.480083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.480363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.481036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:17.482794 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.496155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1151/2294] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:17.502896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.518500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aba03-fa17-4ab6-a4d5-013bda503eaa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:17.526996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205d4cd-8da0-4032-8bab-0aaf95a79960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:17.531609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d4cd-8da0-4032-8bab-0aaf95a79960 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.531609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d4cd-8da0-4032-8bab-0aaf95a79960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.531609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d4cd-8da0-4032-8bab-0aaf95a79960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.531609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205d4cd-8da0-4032-8bab-0aaf95a79960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:17.533469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205d4cd-8da0-4032-8bab-0aaf95a79960 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.561351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1144/2295] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:17.566564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.586137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579ef5-3304-4143-936c-a4dd618ce2e5 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:17.595475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:17.605076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.605076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.605076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] PATH_INFO: `/v3/projects/adf6f1699a0740ee99ae03ef2423d143/users/77b0f8fe6dde4842bb3d03150ebe9916/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.625593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf6f1699a0740ee99ae03ef2423d143, user_id=77b0f8fe6dde4842bb3d03150ebe9916, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:17.632559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.809194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1152/2296] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:17 2022] PUT /identity/v3/projects/adf6f1699a0740ee99ae03ef2423d143/users/77b0f8fe6dde4842bb3d03150ebe9916/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:17.816389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c24ed-2ff2-4d60-821c-32dd3c1ed10e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.817288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c24ed-2ff2-4d60-821c-32dd3c1ed10e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.817288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c24ed-2ff2-4d60-821c-32dd3c1ed10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.857649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0e1c24ed-2ff2-4d60-821c-32dd3c1ed10e None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:17.860243 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0e1c24ed-2ff2-4d60-821c-32dd3c1ed10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:17.976417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1145/2297] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:17.989310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.992648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:18.172569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d1d36-8dd1-4d0e-a688-870e011123dd admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:18.180435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:18.181415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.182012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.182464 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.198596 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:18.200604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.223383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1153/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:27.143529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:27.145427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:27.146231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:27.228078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] MFA Rules not processed for user `77b0f8fe6dde4842bb3d03150ebe9916`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:27.229488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:27.234445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:27.369958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1146/2299] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:27.372904 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:27.386183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce5626-47b1-4f51-b879-553f6e8d7b76 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:27.406002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:27.406002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:27.406002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:27.406002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:27.406719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:27.406719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:27.418092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:27.601753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1154/2300] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:27 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:27.607149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:27.623977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c59a4-549c-48dd-9429-0b92fc467016 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:27.743805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:27.744905 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:27.746102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:27.746744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:27.748630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:27.753126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:27.871145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1147/2301] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:27 2022] POST /identity/v3/users => generated 298 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:27.879179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:27.899644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f97efc-9d18-415f-971f-b6055e8adaac admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:27.906714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:27.907574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:27.907971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:27.908495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:27.909551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:27.912434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:27.929190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1155/2302] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:27.933494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:27.945549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e32e2c-e9a1-4db7-aa5f-408cf7a31572 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:27.951122 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f775-28be-48a8-9a9c-cac88a9e48fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:27.952255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f775-28be-48a8-9a9c-cac88a9e48fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:27.952545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f775-28be-48a8-9a9c-cac88a9e48fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:27.952830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f775-28be-48a8-9a9c-cac88a9e48fe admin admin] PATH_INFO: `/v3/projects/7b23980a30624073a2ce8c34b1f56cc6/users/d3087dab35ee4b918a9df9f2035bda66/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:27.991968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31f775-28be-48a8-9a9c-cac88a9e48fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b23980a30624073a2ce8c34b1f56cc6, user_id=d3087dab35ee4b918a9df9f2035bda66, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:27.997526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31f775-28be-48a8-9a9c-cac88a9e48fe admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:28.159724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1148/2303] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:27 2022] PUT /identity/v3/projects/7b23980a30624073a2ce8c34b1f56cc6/users/d3087dab35ee4b918a9df9f2035bda66/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:28.165244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:28.189815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9c28-ef78-40bb-9e2e-e93cc5b82fff admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:28.275724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:28.279934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:28.279934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:28.289488 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:28.294010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:28.294010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:28.294010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:28.294010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:28.295635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:28.312332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1156/2304] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:28.313599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:28.315996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] MFA Rules not processed for user `7ae66df61a764a42a1b1b15f54d5c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:28.317062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:28.337583 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72d85b-6558-4d0e-bbef-f2aba43ee476 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:28.346294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdfe78-d8cb-4522-be5d-9b08da8d2a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:28.346980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdfe78-d8cb-4522-be5d-9b08da8d2a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:28.347259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdfe78-d8cb-4522-be5d-9b08da8d2a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:28.347547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdfe78-d8cb-4522-be5d-9b08da8d2a77 admin admin] PATH_INFO: `/v3/projects/7b23980a30624073a2ce8c34b1f56cc6/users/d3087dab35ee4b918a9df9f2035bda66/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:28.349704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdfe78-d8cb-4522-be5d-9b08da8d2a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b23980a30624073a2ce8c34b1f56cc6, user_id=d3087dab35ee4b918a9df9f2035bda66, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:28.352493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdfe78-d8cb-4522-be5d-9b08da8d2a77 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:28.378835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1157/2305] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:28 2022] PUT /identity/v3/projects/7b23980a30624073a2ce8c34b1f56cc6/users/d3087dab35ee4b918a9df9f2035bda66/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:28.446042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1149/2306] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:35.533301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89885b8a-d9c6-42a8-84ae-8bc3e3786a7a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:35.534144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89885b8a-d9c6-42a8-84ae-8bc3e3786a7a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:35.534284 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89885b8a-d9c6-42a8-84ae-8bc3e3786a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:35.574635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-89885b8a-d9c6-42a8-84ae-8bc3e3786a7a None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:35.577054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-89885b8a-d9c6-42a8-84ae-8bc3e3786a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:35.670719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1158/2307] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 14:09:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:35.707292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:35.709245 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:35.737302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:35.844360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5be8-67c9-4084-a5a8-d9622ac1ac3d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:35.929300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:35.930446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:35.930798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:35.931144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:35.941283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:35.943255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:35.954677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1150/2308] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:09:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:09:40.087805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8df9be-f802-4694-8578-6fe4bb4028a5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.088752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8df9be-f802-4694-8578-6fe4bb4028a5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.089957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8df9be-f802-4694-8578-6fe4bb4028a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.137455 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-ee8df9be-f802-4694-8578-6fe4bb4028a5 None None] MFA Rules not processed for user `d3087dab35ee4b918a9df9f2035bda66`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:40.139126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ee8df9be-f802-4694-8578-6fe4bb4028a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:40.363135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:40.396377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1159/2309] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:40.420974 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:40.425384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:40.549708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073341f8-237e-4ee2-9a9c-4c16fa2f9ce3 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:40.556056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:40.557130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.558035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.558415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.687016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:40.690991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:40.715169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1151/2310] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:41.191136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af391dc-2d9e-4005-b9fe-657308fe6a15 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:41.192742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af391dc-2d9e-4005-b9fe-657308fe6a15 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:41.193867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af391dc-2d9e-4005-b9fe-657308fe6a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:41.252585 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6af391dc-2d9e-4005-b9fe-657308fe6a15 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:41.253887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6af391dc-2d9e-4005-b9fe-657308fe6a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:41.269940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1160/2311] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:41.288153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:41.299928 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:41.316238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de601-b57e-4352-b974-f0f09dd3bf31 service glance] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:41.326844 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:41.328774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:41.338609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:41.340431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:41.358330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:41.367930 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:41.418619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1152/2312] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:09:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:09:51.243761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=6 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:52.281604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:55.489686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f452e-c6f0-4c05-bbb4-66509e3f708e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.490341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f452e-c6f0-4c05-bbb4-66509e3f708e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.490458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f452e-c6f0-4c05-bbb4-66509e3f708e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.512253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8c5f452e-c6f0-4c05-bbb4-66509e3f708e None None] MFA Rules not processed for user `77b0f8fe6dde4842bb3d03150ebe9916`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:55.513778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8c5f452e-c6f0-4c05-bbb4-66509e3f708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:55.523398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1161/2313] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:04.483866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:04.763411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229e94d-1503-4407-877d-268b04a1fcc2 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:04.769882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:04.770970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:04.771697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:04.772221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:04.775446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:04.890343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:04.892779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:04.906342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1153/2314] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:05.505045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:08.547070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:08.549648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:08.550555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:08.584004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:08.585376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:08.613277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1162/2315] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:10:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:08.614843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:08.638692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40861f8-0d9f-411b-bf7c-80f92432f946 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:08.650871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:08.652388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:08.653041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:08.653620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:08.676552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:08.681880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:08.701520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1154/2316] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:10:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:10:18.573386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:18.693402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=56 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:38.474951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:38.486254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22b13-db2d-44fa-a58e-530a93c7df7c None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:38.493214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:38.494718 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:38.495209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:38.495742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:38.499178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:38.600304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:38.601792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:38.614659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1163/2317] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:10:41.360926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:41.388987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f598360-fa32-4232-990d-22dc3822d636 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:41.403063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:41.404102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:41.404626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:41.405740 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:41.409099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:41.432471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:41.434427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17eb26-cc3d-497f-916f-40fe6d7a3738 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:41.458193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1155/2318] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:47.041526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:47.058038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2076-a41c-4731-9526-a25f392d7352 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:47.063082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a081a1-06a9-4441-957f-21fe3fe6017b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:47.064862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a081a1-06a9-4441-957f-21fe3fe6017b service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:47.065317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a081a1-06a9-4441-957f-21fe3fe6017b service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:47.065768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a081a1-06a9-4441-957f-21fe3fe6017b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:47.080481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a081a1-06a9-4441-957f-21fe3fe6017b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:47.083334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a081a1-06a9-4441-957f-21fe3fe6017b service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:47.100220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1164/2319] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:49.355200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:49.358200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:49.359515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:49.406200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] MFA Rules not processed for user `e53d275b44a2432a9fe8611304bcfa91`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:49.408054 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:49.486982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1156/2320] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:49.518370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:49.520548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:49.521264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:49.583441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] MFA Rules not processed for user `61401557e7624580b20de6e31c92f785`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:49.585840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:49.681416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1165/2321] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:59.471862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=31 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:59.673408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=65 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:59.991913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:59.992160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:00.028647 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3df4c-aa07-47fa-83e7-3ff627fe594a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:00.055987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:00.059875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:00.059875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:00.059875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:00.067493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:00.094400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:00.096375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b37c5-c0bf-4ffd-90d8-d1b64fa85195 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:00.120984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1157/2322] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:07.421249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:07.424879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:07.443395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05b79-7b74-450a-8a51-6daa1a6e0756 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:07.450193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:07.450833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:07.451143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:07.451374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:07.453556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:07.557239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:07.559422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:07.574851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1166/2323] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:10.103002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:11.140911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.143482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.143945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.177538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] MFA Rules not processed for user `7562a0429fdf42948174b517b211884f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:11.186212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:11.208275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1158/2324] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:11:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:11:12.520851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:12.547785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddb756-c3a2-4857-a942-74b37a8fe88a service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:12.553598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:12.555282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:12.555760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:12.556203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:12.559669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:12.624246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:12.624246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9c22b-2f8d-4761-93b9-35101cb1fdf0 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:12.632638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1167/2325] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:17.627914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:17.630391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:17.767312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8b509-276e-4bd0-8208-0b0707d07ae8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:17.771852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:17.772847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:17.773286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:17.773577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:17.776699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:17.929168 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:17.931814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:17.955701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1159/2326] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:22.625420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=34 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:25.226735 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:25.230346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:25.230346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:25.262536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] MFA Rules not processed for user `7ae66df61a764a42a1b1b15f54d5c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:25.264636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:25.351672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1168/2327] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:11:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:25.866725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:25.890669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180529-aa42-43f9-bfcc-33463e820a96 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:25.897619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:25.898796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:25.899562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:25.900473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:25.904072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:25.919360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:25.921042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db84be-1529-4c9b-809f-38cdbfa65511 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:25.936989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1160/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:26.735715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:26.737581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:26.852514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47e670-4269-4cf3-9593-3fce536bb22f None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:26.859824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:26.861404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:26.861842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:26.862252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:26.865201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:26.887299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:26.890268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:26.911876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1169/2329] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:35.150379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:35.151015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:35.151335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:35.173957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] MFA Rules not processed for user `ff664c7ba095413daad6b3f1ebc49cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:35.175182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:35.182402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1161/2330] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:35.262639 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:35.278570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051de9a-bb04-4638-b624-b36e8fb8501a service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:35.287151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:35.287151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:35.287633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:35.287633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:35.289739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:35.313348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:35.316369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:35.338188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1170/2331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:41.803545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:41.819410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=61 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:41.830910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcaf02-ba5c-40ef-b4b0-7328b7daca1e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:41.837561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:41.838354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:41.838786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:41.839258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:41.841225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:41.854759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:41.856549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:41.871229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1162/2332] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:44.146775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:44.166425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a688d6-8ebf-429d-9db8-3f1c5df3b6c6 service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:44.174292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:44.175599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:44.176206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:44.177222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:44.180539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:44.202393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:44.205724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:44.226436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1171/2333] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:52.127869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:54.217268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=69 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:16.850389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:16.851938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:17.053705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4513444-e497-4a3c-a391-3be4e93a07b6 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:17.059737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:17.062517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:17.062517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:17.062517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] PATH_INFO: `/v3/users/e53d275b44a2432a9fe8611304bcfa91` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:17.063111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53d275b44a2432a9fe8611304bcfa91)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:17.064784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:17.325380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53d275b44a2432a9fe8611304bcfa91'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:17.326426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53d275b44a2432a9fe8611304bcfa91'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:17.342498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53d275b44a2432a9fe8611304bcfa91'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:17.448427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53d275b44a2432a9fe8611304bcfa91'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:17.462055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b3f34988-d10a-4dc1-9fd7-37db0a3e4278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53d275b44a2432a9fe8611304bcfa91'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:17.489849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1163/2334] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:12:16 2022] DELETE /identity/v3/users/e53d275b44a2432a9fe8611304bcfa91 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:12:17.872799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:17.997190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6fe3f-8e39-4339-a3d4-0dfa4abed9f1 service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:18.128545 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:18.130153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:18.131134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:18.131869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] PATH_INFO: `/v3/projects/2e6d998735c243c3bcd08ca2e2d81ae9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:18.134418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6d998735c243c3bcd08ca2e2d81ae9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:18.138748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:18.288782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6d998735c243c3bcd08ca2e2d81ae9', 'request_id': 'req-03185935-f0a8-4f99-870b-3781637ceae6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:18.304169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6d998735c243c3bcd08ca2e2d81ae9', 'request_id': 'req-03185935-f0a8-4f99-870b-3781637ceae6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:18.413077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6d998735c243c3bcd08ca2e2d81ae9', 'request_id': 'req-03185935-f0a8-4f99-870b-3781637ceae6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:18.439782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] The token cache is being invalidate because project 2e6d998735c243c3bcd08ca2e2d81ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:12:18.440644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:18.442037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-03185935-f0a8-4f99-870b-3781637ceae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:18.445682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1172/2335] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:12:17 2022] DELETE /identity/v3/projects/2e6d998735c243c3bcd08ca2e2d81ae9 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:18.451915 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:18.453305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:18.454119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:18.493149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:18.584166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:18.585776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:18.704359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1164/2336] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:18.714153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:18.715226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:18.715713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:18.746168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:18.747451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:18.757773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1173/2337] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:18.762788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:18.779417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57d16a-e32b-4a91-bf15-783df6673f36 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:18.785466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:18.787423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:18.787423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:18.788679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:18.789545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:12:18.790009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:18.792221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:18.810878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1165/2338] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:12:18.817542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:18.835236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bafa56-c263-4597-b606-211aeea047eb None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:18.846080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:18.846080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:18.846080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:18.846080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:18.847069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:18.851055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:19.004989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1174/2339] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:12:18 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:12:19.008782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:19.019904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a41f71-3fc3-4b57-80f8-ebf48964ea70 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:19.100780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:19.101539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:19.101819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:19.102149 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:19.102804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:19.104744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:19.204186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1166/2340] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:12:19 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:12:19.213038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:19.231810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee01c9-c5db-4b8c-b30d-5a3c0f99450f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:19.240413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:19.241846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:19.242389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:19.242980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:19.244201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:19.247471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:19.272163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1175/2341] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:12:19.274612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:19.292179 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee31936-2715-4c36-9ac4-b7b26c19c6e8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:19.302315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66399-73f5-47cc-bdc1-eaa7e3278f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:19.303945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66399-73f5-47cc-bdc1-eaa7e3278f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:19.304451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66399-73f5-47cc-bdc1-eaa7e3278f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:19.304937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66399-73f5-47cc-bdc1-eaa7e3278f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:19.306186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba66399-73f5-47cc-bdc1-eaa7e3278f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:19.309359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba66399-73f5-47cc-bdc1-eaa7e3278f19 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:19.326778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1167/2342] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:12:19.332482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:19.352260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5ddaf-169f-45a0-a277-7b9818849f6e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:19.361070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:19.362250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:19.362717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:19.363186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] PATH_INFO: `/v3/projects/8acdb976501045c0813d16f3d0ffd13d/users/93f843d15d194f0b9f44ec33c49b9b4c/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:19.393846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8acdb976501045c0813d16f3d0ffd13d, user_id=93f843d15d194f0b9f44ec33c49b9b4c, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:19.399473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:19.406215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:19.652775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1176/2343] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:12:19 2022] PUT /identity/v3/projects/8acdb976501045c0813d16f3d0ffd13d/users/93f843d15d194f0b9f44ec33c49b9b4c/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:19.659992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d032ba-473a-49d9-8a63-6120dc0a6f03 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:19.659992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d032ba-473a-49d9-8a63-6120dc0a6f03 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:19.659992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d032ba-473a-49d9-8a63-6120dc0a6f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:19.684098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e8d032ba-473a-49d9-8a63-6120dc0a6f03 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:19.685645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e8d032ba-473a-49d9-8a63-6120dc0a6f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:19.759658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:19.777791 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1168/2344] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:19.799440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:19.802740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:19.949829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fa207-c653-4aeb-8a51-7f638d3e3b62 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:19.954683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:19.955337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:19.955621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:19.955891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:19.967446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:19.969788 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:19.983156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1177/2345] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:12:29.258164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:29.259003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:29.259003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:29.306255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] MFA Rules not processed for user `93f843d15d194f0b9f44ec33c49b9b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:29.308287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:29.433515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1169/2346] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:29.439331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:29.450654 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458178-94b5-48dc-b6d3-95125adf8d8d service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:29.457995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:29.459763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:29.460329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:29.460814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:29.461751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:29.466832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:29.617052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1178/2347] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:12:29 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:12:29.621772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:29.652489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f4b1d-59d2-4e2e-b1eb-3404aa050f4a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:29.770586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:29.771216 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:29.771496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:29.771771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:29.772413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:29.774463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:30.002171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1170/2348] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:12:29 2022] POST /identity/v3/users => generated 296 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:12:30.004402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:30.025587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956e197-7f2e-4398-bb1a-1592735a3b4c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:30.035304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:30.036701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:30.037175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:30.037871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:30.039076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:30.041737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:30.064384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1179/2349] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:12:30.066910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:30.085551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a277e4-2cc0-4c04-b6d8-8e646cb6da23 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:30.091329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:30.091865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:30.092363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:30.093451 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] PATH_INFO: `/v3/projects/3a8e2734029f4bf49339ca13ef88ee29/users/f9af38df76824077b4133fa9b8150a5f/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:30.120381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8e2734029f4bf49339ca13ef88ee29, user_id=f9af38df76824077b4133fa9b8150a5f, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:30.126389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:30.313815 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1171/2350] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:12:30 2022] PUT /identity/v3/projects/3a8e2734029f4bf49339ca13ef88ee29/users/f9af38df76824077b4133fa9b8150a5f/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:30.320228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:30.339134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db971d-bd2e-495f-973b-d24173d01c41 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:30.459204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44204d74-a220-4242-ac4f-c1abc4aa5865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:30.460460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44204d74-a220-4242-ac4f-c1abc4aa5865 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:30.461392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44204d74-a220-4242-ac4f-c1abc4aa5865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:30.462253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44204d74-a220-4242-ac4f-c1abc4aa5865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:30.463621 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44204d74-a220-4242-ac4f-c1abc4aa5865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:30.467224 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44204d74-a220-4242-ac4f-c1abc4aa5865 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:30.489705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1180/2351] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:12:30.495898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:30.514942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44496e-1f52-40af-b229-e223aa5d4952 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:30.525884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:30.527696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:30.530033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:30.531429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] PATH_INFO: `/v3/projects/3a8e2734029f4bf49339ca13ef88ee29/users/f9af38df76824077b4133fa9b8150a5f/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:30.536822 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8e2734029f4bf49339ca13ef88ee29, user_id=f9af38df76824077b4133fa9b8150a5f, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:30.545177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:30.573607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1172/2352] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:12:30 2022] PUT /identity/v3/projects/3a8e2734029f4bf49339ca13ef88ee29/users/f9af38df76824077b4133fa9b8150a5f/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:32.852626 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:32.853413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:32.853413 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:32.891574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:32.893821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:32.909712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:33.180478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1181/2353] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:12:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:12:33.271666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:33.275861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:33.416085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:33.545426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b62614-3038-48cb-aa92-005a1fc9f3cd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:33.562733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:33.564362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:33.582035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:33.582035 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:33.585576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:33.589304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:33.620859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1173/2354] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:12:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:12:36.173320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:36.175829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:36.434129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf447e-996d-425b-bba5-037bdc3037aa None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:36.456383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:36.460065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:36.460499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:36.460909 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:36.469570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:36.580437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:36.582018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e87d2-ec93-4036-99de-a486977830dd service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:36.607641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1182/2355] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:12:38.147643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:38.154090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:38.401340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34f69-2a83-4a2e-97b0-1ba5d28d89e1 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:38.424813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:38.425486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:38.425776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:38.426097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:38.433678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:38.664957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:38.669855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:38.698085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1174/2356] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:46.588213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=74 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:48.686999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=66 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:50.653417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214496-0fe8-4528-93b2-8163ff839b2a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:50.654160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214496-0fe8-4528-93b2-8163ff839b2a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:50.654160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214496-0fe8-4528-93b2-8163ff839b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:50.726750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-bb214496-0fe8-4528-93b2-8163ff839b2a None None] MFA Rules not processed for user `f9af38df76824077b4133fa9b8150a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:50.730490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bb214496-0fe8-4528-93b2-8163ff839b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:51.124159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1183/2357] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:51.426624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:51.511302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b0593-2154-404f-b9b0-0113fd41504e service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:51.535743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:51.537836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:51.548602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:51.551472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:51.742404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:51.747371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:51.780067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1175/2358] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:54.757987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:54.759483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:54.760212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:54.887410 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:54.893594 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:54.911369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1184/2359] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 14:12:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:54.960407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:55.009651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f638cc-558f-4ba3-9a71-eca88394cc47 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:55.035533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:55.040503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:55.041938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:55.041938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:55.116858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:55.129418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:55.180401 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1176/2360] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:12:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:12:55.815476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:55.820037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:56.043782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717119e-0bd7-4a95-8049-005de2cc525a None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:56.055146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:56.059118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:56.059870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:56.060487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:56.065730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:56.093426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:56.097197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:56.144499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1185/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:56.207759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:56.217613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:56.260970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cc38b-9ee3-4360-90f3-c3d2de8d56ce service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:56.273955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:56.276082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:56.276944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:56.277602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] PATH_INFO: `/v3/users/7562a0429fdf42948174b517b211884f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:56.279879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7562a0429fdf42948174b517b211884f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:56.290013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:56.703479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7562a0429fdf42948174b517b211884f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:56.707307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7562a0429fdf42948174b517b211884f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:56.761417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7562a0429fdf42948174b517b211884f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:56.882433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7562a0429fdf42948174b517b211884f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:56.896858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5820d6a6-30e6-4823-87b6-5d12b3e045e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7562a0429fdf42948174b517b211884f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:56.907853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1177/2362] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:12:56 2022] DELETE /identity/v3/users/7562a0429fdf42948174b517b211884f => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:57.699129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:57.749842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458f22-5198-469e-bddf-28c2ca0ec0fe service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:58.197121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:58.198179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:58.198701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:58.199222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] PATH_INFO: `/v3/projects/36049d2090b84443a3bf3bf00558379e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:58.207660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36049d2090b84443a3bf3bf00558379e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:58.223634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:58.266002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:58.550350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36049d2090b84443a3bf3bf00558379e', 'request_id': 'req-f98611ea-cc03-48dc-b743-fa2535fd5322'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:58.581878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36049d2090b84443a3bf3bf00558379e', 'request_id': 'req-f98611ea-cc03-48dc-b743-fa2535fd5322'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:58.722945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36049d2090b84443a3bf3bf00558379e', 'request_id': 'req-f98611ea-cc03-48dc-b743-fa2535fd5322'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:58.772243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] The token cache is being invalidate because project 36049d2090b84443a3bf3bf00558379e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:12:58.772772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:58.773988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f98611ea-cc03-48dc-b743-fa2535fd5322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:58.782870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1186/2363] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:12:57 2022] DELETE /identity/v3/projects/36049d2090b84443a3bf3bf00558379e => generated 0 bytes in 1079 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:12:58.786836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:58.787872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:58.790345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:58.839001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:58.842181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:59.118444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:59.167980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1178/2364] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:12:59.170371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:59.171634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:59.185086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:59.276040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:59.301642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:59.318406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1187/2365] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:59.323719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:59.349348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28397-d1c6-455a-b538-871bde66d872 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:59.364551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:59.365280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:59.365593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:59.365861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:59.366660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:12:59.367174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:59.369544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:59.396559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1179/2366] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:12:59.401757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:59.539078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d5a8-c35f-4e5d-b1c9-a49a60c242c2 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:59.539078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:59.539078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:59.539078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:59.540266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:59.540266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:59.540266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:00.347466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1188/2367] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:12:59 2022] POST /identity/v3/projects => generated 373 bytes in 947 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:00.353615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:00.378495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dfcb2-534f-4343-878a-b483a9108f09 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:00.632876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:00.636720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:00.637281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:00.637769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:00.646299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:00.650924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:01.340670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1180/2368] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:00 2022] POST /identity/v3/users => generated 287 bytes in 983 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:01.343127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:01.378396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bffeb9-bb32-44c6-af8c-272612c7021d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:01.391093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:01.393679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:01.393679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:01.394733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:01.396701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:01.401669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:01.432912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1189/2369] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:01.433570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:01.458862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4e2ce-eb4f-4555-aadd-f5c674f80dc6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:01.475469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a15e4-7fd1-4b55-8212-a5a43894384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:01.476116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a15e4-7fd1-4b55-8212-a5a43894384a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:01.476405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a15e4-7fd1-4b55-8212-a5a43894384a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:01.476685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a15e4-7fd1-4b55-8212-a5a43894384a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:01.477430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a15e4-7fd1-4b55-8212-a5a43894384a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:01.482454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a15e4-7fd1-4b55-8212-a5a43894384a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:01.505640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1181/2370] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:01.507867 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:01.549659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a32f-260d-450f-86db-57b68541ad97 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:01.580759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:01.584905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:01.586218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:01.588204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] PATH_INFO: `/v3/projects/9320dd382479483bb7b455a9801daf03/users/c20db8c34b624787939432ebb4655978/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:01.683434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9320dd382479483bb7b455a9801daf03, user_id=c20db8c34b624787939432ebb4655978, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:01.695080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:01.882175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1190/2371] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:01 2022] PUT /identity/v3/projects/9320dd382479483bb7b455a9801daf03/users/c20db8c34b624787939432ebb4655978/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:01.908427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:01.909163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:01.909636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:02.050217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] MFA Rules not processed for user `c20db8c34b624787939432ebb4655978`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:02.061214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:02.340682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1182/2372] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:02.493203 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:02.499681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:02.818821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d896e8e-3801-4dbb-aec8-9000c252165d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:02.836318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca4d8-2bc8-4878-b3c4-dcbc0a6ef626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:02.836967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca4d8-2bc8-4878-b3c4-dcbc0a6ef626 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:02.841388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca4d8-2bc8-4878-b3c4-dcbc0a6ef626 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:02.841388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca4d8-2bc8-4878-b3c4-dcbc0a6ef626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:02.854905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ca4d8-2bc8-4878-b3c4-dcbc0a6ef626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:02.861812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ca4d8-2bc8-4878-b3c4-dcbc0a6ef626 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:02.899312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1191/2373] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:07.857053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:07.950850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:07.965608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177af89-2f9c-43b6-bfd2-d703f485882f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:07.971167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:07.971831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:07.972118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:07.972403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] PATH_INFO: `/v3/users/c20db8c34b624787939432ebb4655978` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:07.973683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20db8c34b624787939432ebb4655978)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:07.975719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:08.258656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20db8c34b624787939432ebb4655978'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:08.260213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20db8c34b624787939432ebb4655978'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:08.298442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20db8c34b624787939432ebb4655978'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:08.417967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20db8c34b624787939432ebb4655978'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:08.431581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20db8c34b624787939432ebb4655978'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:08.441318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1183/2374] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:07 2022] DELETE /identity/v3/users/c20db8c34b624787939432ebb4655978 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:08.447726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:08.448561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:08.448561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:08.475942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:08.477401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:08.501279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:08.558967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1192/2375] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:08.572902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:08.582693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:08.680959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794546d6-6f87-4d5f-bfca-bf41c7279c2b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:08.686817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4bcff-6da5-47ce-a593-6fd5856c81a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:08.688308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4bcff-6da5-47ce-a593-6fd5856c81a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:08.688705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4bcff-6da5-47ce-a593-6fd5856c81a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:08.688987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4bcff-6da5-47ce-a593-6fd5856c81a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:08.704531 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4bcff-6da5-47ce-a593-6fd5856c81a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:08.706120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4bcff-6da5-47ce-a593-6fd5856c81a9 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:08.717440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1184/2376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:09.563967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:09.567874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:09.608770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d6fc-eab2-4df3-84a1-5137da798965 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:09.626120 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:09.627283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:09.630442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:09.630898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] PATH_INFO: `/v3/projects/9320dd382479483bb7b455a9801daf03` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:09.636082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9320dd382479483bb7b455a9801daf03)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:09.641783 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:09.883684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9320dd382479483bb7b455a9801daf03', 'request_id': 'req-4827350b-aea5-433b-9517-19261720d713'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.895690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9320dd382479483bb7b455a9801daf03', 'request_id': 'req-4827350b-aea5-433b-9517-19261720d713'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.007952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9320dd382479483bb7b455a9801daf03', 'request_id': 'req-4827350b-aea5-433b-9517-19261720d713'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.041723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] The token cache is being invalidate because project 9320dd382479483bb7b455a9801daf03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:10.042925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.044777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4827350b-aea5-433b-9517-19261720d713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.049869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1193/2377] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:09 2022] DELETE /identity/v3/projects/9320dd382479483bb7b455a9801daf03 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:10.056911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d369d-ab70-41da-a391-833783b74c9a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:10.057874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d369d-ab70-41da-a391-833783b74c9a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:10.058462 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d369d-ab70-41da-a391-833783b74c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:10.110805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-900d369d-ab70-41da-a391-833783b74c9a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:10.114673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-900d369d-ab70-41da-a391-833783b74c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:10.369939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1185/2378] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:10.380062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:10.380894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:10.381755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:10.442339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:10.444219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:10.458961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1194/2379] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:10.460126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d369d-ab70-41da-a391-833783b74c9a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:10.477837 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d369d-ab70-41da-a391-833783b74c9a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:10.495818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:10.496960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:10.497475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:10.498680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:10.500024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:13:10.500393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:10.502820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:10.531784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1186/2380] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:13:10.536375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:10.549745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cfd70-5d5b-42b6-888f-0b8f49d6d490 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:10.557922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:10.559024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:10.559550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:10.560264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:10.562193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:10.569320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:10.839084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:10.840082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1195/2381] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:10 2022] POST /identity/v3/projects => generated 383 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:10.845767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:10.855350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904aef2-3730-4bcd-a15e-d7b4a2b4c384 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:11.002252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:11.002899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:11.003192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:11.003516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:11.005148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:11.007256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:11.152186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1187/2382] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:10 2022] POST /identity/v3/users => generated 292 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:11.159853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:11.182437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c848c7-b4fc-49be-9986-160a9b4ff6b4 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:11.205337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:11.206842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:11.210134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:11.211613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:11.212852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:11.217661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:11.246043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1196/2383] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:11.249989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:11.267273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e64c-e4b8-4962-8d49-53d8464dde97 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:11.281229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2199182-b529-4016-b4dc-5a03fad106ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:11.282026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199182-b529-4016-b4dc-5a03fad106ae admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:11.283341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199182-b529-4016-b4dc-5a03fad106ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:11.284365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199182-b529-4016-b4dc-5a03fad106ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:11.287163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199182-b529-4016-b4dc-5a03fad106ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:11.294187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199182-b529-4016-b4dc-5a03fad106ae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:11.333563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1188/2384] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:11.340024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:11.357177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6a1cf-c019-4f05-84ea-6e8080d30d39 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:11.369003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:11.369717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:11.370029 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:11.372537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] PATH_INFO: `/v3/projects/e6e2778c988b462cbfbdcfbd62f18072/users/01412c23fb634f32b5c24c498984c2e0/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:11.422142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e2778c988b462cbfbdcfbd62f18072, user_id=01412c23fb634f32b5c24c498984c2e0, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:11.422142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:11.576340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1197/2385] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:11 2022] PUT /identity/v3/projects/e6e2778c988b462cbfbdcfbd62f18072/users/01412c23fb634f32b5c24c498984c2e0/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:11.604514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:11.604804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:11.605483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:11.680523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] MFA Rules not processed for user `01412c23fb634f32b5c24c498984c2e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:11.683918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:11.945578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1189/2386] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:11.954363 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:12.080912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:12.137513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0fec8-2dfc-4986-a699-d9df534ca1a7 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:12.148979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:12.150004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:12.150663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:12.151289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:12.152474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:12.156524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:12.329178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1198/2387] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:11 2022] POST /identity/v3/projects => generated 385 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:12.335101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:12.358700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1be36-f443-4a7f-ae3d-ca25eaa675ef None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:12.500425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:12.501813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:12.502549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:12.503373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:12.504875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:12.509521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:12.677942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1190/2388] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:12 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:12.679736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:12.693340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a629c7-5a56-4f02-8487-201497d70d52 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:12.700307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:12.701084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:12.701438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:12.702017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:12.702770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:12.704640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:12.725764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1199/2389] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:12.730167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:12.742999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b0f79-918d-4dca-aedc-982fb165824f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:12.750899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11ab63-01d1-4564-a50b-b5b068000c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:12.751710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11ab63-01d1-4564-a50b-b5b068000c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:12.752098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11ab63-01d1-4564-a50b-b5b068000c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:12.752475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11ab63-01d1-4564-a50b-b5b068000c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:12.753239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11ab63-01d1-4564-a50b-b5b068000c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:12.755194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11ab63-01d1-4564-a50b-b5b068000c94 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:12.768375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1191/2390] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:12.776003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:12.796005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32a3c-3d56-45f1-9a0c-a8e76301437a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:12.809802 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:12.812557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:12.813311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:12.814037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] PATH_INFO: `/v3/projects/3c9a944752874718a1f2f6ec13248968/users/467fab876866479697db31e7bb2de4bc/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:12.877123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9a944752874718a1f2f6ec13248968, user_id=467fab876866479697db31e7bb2de4bc, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:12.894738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:13.056501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1200/2391] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:12 2022] PUT /identity/v3/projects/3c9a944752874718a1f2f6ec13248968/users/467fab876866479697db31e7bb2de4bc/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:13.076664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:13.076664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:13.076664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:13.084141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:13.185141 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] MFA Rules not processed for user `467fab876866479697db31e7bb2de4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:13.195409 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:13.354627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:13.359024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1192/2392] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:13.526277 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f35cc5-2525-4b36-ba10-2c00c235fb71 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:13.532651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:13.533381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:13.533733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:13.534151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:13.534884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:13.536934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:13.673343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1201/2393] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:13 2022] POST /identity/v3/projects => generated 387 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:13.675400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:13.726252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:13.728678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:13.731038 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226fd44-7728-458e-86f6-97720ab2ac3f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.002476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.002476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.002968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.003463 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.005917 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.012871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.060064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2032a1-9d48-4e95-b155-d32d73591ce1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.071025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.072668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.073231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.073835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.076538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:14.106042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.114176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.159955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1202/2394] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:14.172271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:14.178955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:14.220265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:14.228845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b67e8-3579-422d-bb36-0c844afefd00 service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.234912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1193/2395] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:13 2022] POST /identity/v3/users => generated 300 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:14.242153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:14.243122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.244194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.244700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.245395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] PATH_INFO: `/v3/users/77b0f8fe6dde4842bb3d03150ebe9916` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.248370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b0f8fe6dde4842bb3d03150ebe9916)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.253385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.300674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3d478-74b8-4e7a-b37f-ff5abdbc8c07 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.309498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.316265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.316265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.316265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.316265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.317225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.339847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1194/2396] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:14.346488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:14.366729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ba090-132f-4839-82d2-b4f15f427023 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.382603 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.383924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.384690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.385007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] PATH_INFO: `/v3/projects/6b773c6a12304414ad48b2c66da4728a/users/61dcbf5bfa5649e7bf67f837fe1fbe68/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.410063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b773c6a12304414ad48b2c66da4728a, user_id=61dcbf5bfa5649e7bf67f837fe1fbe68, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.412807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.609041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1195/2397] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:14 2022] PUT /identity/v3/projects/6b773c6a12304414ad48b2c66da4728a/users/61dcbf5bfa5649e7bf67f837fe1fbe68/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:14.617333 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:14.658223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b0f8fe6dde4842bb3d03150ebe9916'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:14.716228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:14.773045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502aba23-0eeb-4730-acc5-103bb1eec4e5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.777598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b0f8fe6dde4842bb3d03150ebe9916'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:14.784939 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.786123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.786621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.787145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.788576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.792060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.799196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b0f8fe6dde4842bb3d03150ebe9916'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:14.799980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b0f8fe6dde4842bb3d03150ebe9916'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:14.811256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2f15c315-a9e1-45fa-847d-364a4581036d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b0f8fe6dde4842bb3d03150ebe9916'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:14.856348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1196/2398] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:14.856459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1203/2399] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:14 2022] DELETE /identity/v3/users/77b0f8fe6dde4842bb3d03150ebe9916 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:14.856566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:14.856566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:14.856566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141306f0-fd03-4dab-906d-74601ae640a2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:14.917301 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:14.918046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.918377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.918688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] PATH_INFO: `/v3/projects/6b773c6a12304414ad48b2c66da4728a/users/61dcbf5bfa5649e7bf67f837fe1fbe68/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:14.921147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b773c6a12304414ad48b2c66da4728a, user_id=61dcbf5bfa5649e7bf67f837fe1fbe68, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:14.925084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:14.943255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1197/2400] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:14 2022] PUT /identity/v3/projects/6b773c6a12304414ad48b2c66da4728a/users/61dcbf5bfa5649e7bf67f837fe1fbe68/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:14.966827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:14.968683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:14.969265 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:15.006560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] MFA Rules not processed for user `61dcbf5bfa5649e7bf67f837fe1fbe68`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:15.008226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:15.118757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1204/2401] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:15.131460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:15.133733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:15.241574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:15.244086 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:15.321838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cadb9b-d7f2-40ed-93ca-5de55930ad30 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:15.337834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:15.338677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:15.340374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:15.340729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:15.376016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:15.378575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:15.392133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1198/2402] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:15.447884 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f93d1-b9b2-4ac1-bcbb-96f213afa4cc None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:15.466238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:15.466238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:15.466238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:15.466238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] PATH_INFO: `/v3/projects/adf6f1699a0740ee99ae03ef2423d143` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:15.467451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf6f1699a0740ee99ae03ef2423d143)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:15.467451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:15.480132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:15.661118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf6f1699a0740ee99ae03ef2423d143', 'request_id': 'req-a2a5161c-5921-48b0-b23c-0094b5f4595b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:15.669640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf6f1699a0740ee99ae03ef2423d143', 'request_id': 'req-a2a5161c-5921-48b0-b23c-0094b5f4595b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:15.775204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf6f1699a0740ee99ae03ef2423d143', 'request_id': 'req-a2a5161c-5921-48b0-b23c-0094b5f4595b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:15.797821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] The token cache is being invalidate because project adf6f1699a0740ee99ae03ef2423d143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:15.798808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:15.801229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a2a5161c-5921-48b0-b23c-0094b5f4595b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:15.804316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1205/2403] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:15 2022] DELETE /identity/v3/projects/adf6f1699a0740ee99ae03ef2423d143 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:15.809235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:15.832497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc409e-e44d-4fd1-99b1-aa7e8f8ef84e service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:15.898524 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:15.922278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:15.923154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:15.923507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:15.923831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] PATH_INFO: `/v3/users/d3087dab35ee4b918a9df9f2035bda66` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:15.926358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3087dab35ee4b918a9df9f2035bda66)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:15.928798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:16.059658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39596ee9-ed99-44ea-80d3-3258c42016c8 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:16.060162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39596ee9-ed99-44ea-80d3-3258c42016c8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:16.062681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39596ee9-ed99-44ea-80d3-3258c42016c8 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:16.066523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1206/2404] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:13:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:13:16.079778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:16.079778 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:16.080716 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:16.163717 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:16.181227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:16.286281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3087dab35ee4b918a9df9f2035bda66'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:16.287166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3087dab35ee4b918a9df9f2035bda66'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:16.318036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3087dab35ee4b918a9df9f2035bda66'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:16.318383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1207/2405] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:16.344640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:16.346980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:16.402155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3087dab35ee4b918a9df9f2035bda66'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:16.421914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3087dab35ee4b918a9df9f2035bda66'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:16.434800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1199/2406] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:15 2022] DELETE /identity/v3/users/d3087dab35ee4b918a9df9f2035bda66 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:16.463084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78db0-d4fb-4fca-b859-1ca5c7aa1f1b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:16.537197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:16.565534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82719025-3a35-4088-8189-54063b32d555 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:16.567279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82719025-3a35-4088-8189-54063b32d555 service glance] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:16.567279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82719025-3a35-4088-8189-54063b32d555 service glance] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:16.567279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82719025-3a35-4088-8189-54063b32d555 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:16.568898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-82719025-3a35-4088-8189-54063b32d555 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:16.680500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82719025-3a35-4088-8189-54063b32d555 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:16.682014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82719025-3a35-4088-8189-54063b32d555 service glance] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:16.695407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1208/2407] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:16.788549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:16.790892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:16.882234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8a6d4-4379-4688-94b0-efc416337021 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:16.887282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:16.887941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:16.888225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:16.888534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] PATH_INFO: `/v3/projects/7b23980a30624073a2ce8c34b1f56cc6` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:16.890336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b23980a30624073a2ce8c34b1f56cc6)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:16.894820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:16.974765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf12b-a1f2-4b9a-b325-285f8b98cea1 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:16.974765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf12b-a1f2-4b9a-b325-285f8b98cea1 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:16.974765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf12b-a1f2-4b9a-b325-285f8b98cea1 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:16.974765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1209/2408] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:13:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:13:16.981187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:16.981822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:16.982987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:17.021210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] MFA Rules not processed for user `2eeb9414ed49428684b9028c18476bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:17.023386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.101959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b23980a30624073a2ce8c34b1f56cc6', 'request_id': 'req-4d810b06-a2ec-4307-b127-769f745078c4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:17.110055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1210/2409] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:17.119137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b23980a30624073a2ce8c34b1f56cc6', 'request_id': 'req-4d810b06-a2ec-4307-b127-769f745078c4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:17.128135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b23980a30624073a2ce8c34b1f56cc6', 'request_id': 'req-4d810b06-a2ec-4307-b127-769f745078c4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:17.135261 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:17.136589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:17.137986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.208616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a589e-fbca-4a29-8de4-36f9e75202f7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:17.282773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] The token cache is being invalidate because project 7b23980a30624073a2ce8c34b1f56cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:17.285982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:17.285982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:17.289059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1200/2410] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:16 2022] DELETE /identity/v3/projects/7b23980a30624073a2ce8c34b1f56cc6 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:17.291821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:17.295241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.333813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:17.335367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:17.336171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:17.336831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:17.340819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.448041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:17.449609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f448b-0df4-44a2-b291-290023d8417b service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:17.462233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1211/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:17.586390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d810b06-a2ec-4307-b127-769f745078c4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:17.591571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:17.592648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:17.593236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:17.593759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:17.602198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.617133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:17.618673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74ba9-42f2-4157-af2e-ca8c3921eb78 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:17.631100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1201/2412] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:17.750041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90e7f7-4275-4882-a15d-6e07825b3649 None None] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:17.750384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90e7f7-4275-4882-a15d-6e07825b3649 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:17.750563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90e7f7-4275-4882-a15d-6e07825b3649 None None] PATH_INFO: `/` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:17.752263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1212/2413] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 14:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:13:17.758464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911cfe1-472d-48b0-a38e-6b3655498471 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:17.759172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911cfe1-472d-48b0-a38e-6b3655498471 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:17.759796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911cfe1-472d-48b0-a38e-6b3655498471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:17.783468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c911cfe1-472d-48b0-a38e-6b3655498471 None None] MFA Rules not processed for user `cf5c9c7caaf4451a9b1ca1a1095408d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:17.784796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c911cfe1-472d-48b0-a38e-6b3655498471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.886586 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1202/2414] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 14:13:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:17.902704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90e7f7-4275-4882-a15d-6e07825b3649 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:17.906709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bb90e7f7-4275-4882-a15d-6e07825b3649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:17.936244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90e7f7-4275-4882-a15d-6e07825b3649 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:17.943495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449f4a2-281e-4cf7-b9fa-edf9f4fc2e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:17.944635 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449f4a2-281e-4cf7-b9fa-edf9f4fc2e9b service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:17.945121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449f4a2-281e-4cf7-b9fa-edf9f4fc2e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:17.945619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449f4a2-281e-4cf7-b9fa-edf9f4fc2e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:17.967337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449f4a2-281e-4cf7-b9fa-edf9f4fc2e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:17.970084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449f4a2-281e-4cf7-b9fa-edf9f4fc2e9b service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:17.990890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1213/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:18.071608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:20.338717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:20.339439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:20.339642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:20.375568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] MFA Rules not processed for user `7ae66df61a764a42a1b1b15f54d5c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:20.377733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:20.491175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1203/2416] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:20.882333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ed0ce-a586-4461-8f0c-1f13498bdf94 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:20.882996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ed0ce-a586-4461-8f0c-1f13498bdf94 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:20.882996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ed0ce-a586-4461-8f0c-1f13498bdf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:20.905280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-de3ed0ce-a586-4461-8f0c-1f13498bdf94 None None] MFA Rules not processed for user `93f843d15d194f0b9f44ec33c49b9b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:20.906698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-de3ed0ce-a586-4461-8f0c-1f13498bdf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:20.938686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:20.950402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645196b-84cf-4480-ab73-d893fa56dd8b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:20.957940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:20.958472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:20.958926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:20.959420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:20.962410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:20.975553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:20.987667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:20.987667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1204/2417] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:20.988562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1214/2418] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:23.619308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:23.621731 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:23.644628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a8a8-8092-4fb5-b6bf-3a81ba56e61e service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:23.659499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:23.660550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:23.660983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:23.661816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:23.664222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:23.699376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:23.810927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:23.812868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:23.840566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1205/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:24.005098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:24.005710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:24.006390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:24.032219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:24.033609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:24.166524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1215/2420] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:13:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:24.179691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:24.182509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:24.385816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ada90-96e9-4d25-89fe-758954e33c0e service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:24.410564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbf5d-0424-4bb4-9a0d-ee01e4a2a3d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:24.411467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbf5d-0424-4bb4-9a0d-ee01e4a2a3d4 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:24.411769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbf5d-0424-4bb4-9a0d-ee01e4a2a3d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:24.412146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbf5d-0424-4bb4-9a0d-ee01e4a2a3d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:24.427008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbf5d-0424-4bb4-9a0d-ee01e4a2a3d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:24.429001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbf5d-0424-4bb4-9a0d-ee01e4a2a3d4 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:24.445667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1206/2421] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:13:29.843062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:29.844591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:29.936668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:30.005540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bc96b-e090-4cf2-8069-6f8af53c1baf None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:30.011797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:30.018139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:30.018139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:30.018139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] PATH_INFO: `/v3/users/01412c23fb634f32b5c24c498984c2e0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:30.018139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01412c23fb634f32b5c24c498984c2e0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:30.020678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:30.321681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01412c23fb634f32b5c24c498984c2e0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:30.467762 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01412c23fb634f32b5c24c498984c2e0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:30.516264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01412c23fb634f32b5c24c498984c2e0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:30.517939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01412c23fb634f32b5c24c498984c2e0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:30.544374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01412c23fb634f32b5c24c498984c2e0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:30.556803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1216/2422] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:29 2022] DELETE /identity/v3/users/01412c23fb634f32b5c24c498984c2e0 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:30.563804 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:30.564396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:30.564788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:30.625120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:30.626555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:30.769134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1207/2423] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:30.777225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:30.780214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:30.900530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a636-1fa7-4dda-8ae5-9948227ae879 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:30.904919 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:30.922294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:30.922294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:30.922294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:30.922294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:30.922294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:30.950308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1217/2424] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:31.702959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:31.707218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:31.724958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef7647-a110-40d5-9263-1d31649e39a1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:31.730773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:31.732550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:31.733133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:31.734178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:31.737732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:31.761378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:31.764681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:31.785971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1208/2425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:31.876126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:31.878543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:31.948641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae3c3-6d4a-4bb2-b1ad-ffd78fd2c2fb service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:31.957630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:31.959072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:31.959404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:31.959667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] PATH_INFO: `/v3/projects/e6e2778c988b462cbfbdcfbd62f18072` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:31.969348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e2778c988b462cbfbdcfbd62f18072)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:31.971688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:32.271349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e2778c988b462cbfbdcfbd62f18072', 'request_id': 'req-0bea4f6f-0506-46da-8562-c9ae9abd14ba'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:32.290523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e2778c988b462cbfbdcfbd62f18072', 'request_id': 'req-0bea4f6f-0506-46da-8562-c9ae9abd14ba'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:32.295641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:32.654713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e2778c988b462cbfbdcfbd62f18072', 'request_id': 'req-0bea4f6f-0506-46da-8562-c9ae9abd14ba'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:32.674779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] The token cache is being invalidate because project e6e2778c988b462cbfbdcfbd62f18072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:32.675522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:32.677076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:32.680196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1218/2426] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:31 2022] DELETE /identity/v3/projects/e6e2778c988b462cbfbdcfbd62f18072 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:32.687378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:32.720479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c64bf-c4f1-4028-af69-4542556fcb51 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:32.808596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:32.870551 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:32.871669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:32.872992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:32.873506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] PATH_INFO: `/v3/users/467fab876866479697db31e7bb2de4bc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:32.875758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=467fab876866479697db31e7bb2de4bc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:32.879643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:33.140879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467fab876866479697db31e7bb2de4bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:33.141696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467fab876866479697db31e7bb2de4bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:33.163701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467fab876866479697db31e7bb2de4bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:33.448296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467fab876866479697db31e7bb2de4bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:33.461494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467fab876866479697db31e7bb2de4bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:33.479299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1209/2427] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:32 2022] DELETE /identity/v3/users/467fab876866479697db31e7bb2de4bc => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:34.037508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:34.040938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:34.082459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea4f6f-0506-46da-8562-c9ae9abd14ba admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:34.203977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:34.211149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:34.211149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:34.211149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] PATH_INFO: `/v3/projects/3c9a944752874718a1f2f6ec13248968` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:34.211945 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9a944752874718a1f2f6ec13248968)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:34.216256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:34.387962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9a944752874718a1f2f6ec13248968', 'request_id': 'req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:34.400558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9a944752874718a1f2f6ec13248968', 'request_id': 'req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:34.509727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9a944752874718a1f2f6ec13248968', 'request_id': 'req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:34.528829 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] The token cache is being invalidate because project 3c9a944752874718a1f2f6ec13248968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:34.529553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:34.530326 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:34.532082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1219/2428] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:34 2022] DELETE /identity/v3/projects/3c9a944752874718a1f2f6ec13248968 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:34.536886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:34.539305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:34.555318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ad88e-eced-49c8-93b2-5ebe374ee393 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:34.659398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:34.660270 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:34.660955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:34.661353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] PATH_INFO: `/v3/users/61dcbf5bfa5649e7bf67f837fe1fbe68` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:34.663596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61dcbf5bfa5649e7bf67f837fe1fbe68)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:34.665952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:35.075284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61dcbf5bfa5649e7bf67f837fe1fbe68'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:35.077402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61dcbf5bfa5649e7bf67f837fe1fbe68'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:35.098513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:35.116584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61dcbf5bfa5649e7bf67f837fe1fbe68'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:35.286442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61dcbf5bfa5649e7bf67f837fe1fbe68'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:35.310510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-51aec630-bbd8-4fe2-8839-f1b821ea8560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61dcbf5bfa5649e7bf67f837fe1fbe68'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:35.336739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1210/2429] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:34 2022] DELETE /identity/v3/users/61dcbf5bfa5649e7bf67f837fe1fbe68 => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:36.762520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:36.762520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:36.785420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a283be-e9ac-4f85-a59e-f4e5cec02bd4 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:36.880622 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:36.881551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:36.881844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:36.882169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] PATH_INFO: `/v3/projects/6b773c6a12304414ad48b2c66da4728a` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:36.883697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b773c6a12304414ad48b2c66da4728a)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:36.886034 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:36.924445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:37.186744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b773c6a12304414ad48b2c66da4728a', 'request_id': 'req-9fd172d1-2d46-4698-a73c-2555c4863fc9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:37.209532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b773c6a12304414ad48b2c66da4728a', 'request_id': 'req-9fd172d1-2d46-4698-a73c-2555c4863fc9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:37.325670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b773c6a12304414ad48b2c66da4728a', 'request_id': 'req-9fd172d1-2d46-4698-a73c-2555c4863fc9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:37.342333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] The token cache is being invalidate because project 6b773c6a12304414ad48b2c66da4728a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:37.342333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:37.349490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9fd172d1-2d46-4698-a73c-2555c4863fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:37.349490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1220/2430] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:36 2022] DELETE /identity/v3/projects/6b773c6a12304414ad48b2c66da4728a => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:37.352197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:37.352932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:37.352932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:37.387625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:37.389262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:37.617575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1211/2431] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:37.618566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:37.619702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:37.620253 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:37.673776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:37.679656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:37.701954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1221/2432] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:37.709851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:37.749648 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea0c95-adcf-4961-8c4b-5daef5f0aeef None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:37.788909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:37.790072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:37.790583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:37.791068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:37.792957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:13:37.793537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:37.799282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:37.852780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1212/2433] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:13:37.854746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:37.885312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f8cbe-c80b-4ee3-9063-6af9ece860c5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:37.916753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:37.918177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:37.918824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:37.921183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:37.922891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:37.932283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:38.111950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1222/2434] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:37 2022] POST /identity/v3/projects => generated 383 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:38.117567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:38.137267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf22f0-5b6d-49e0-a5e7-5de01986fb21 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:38.277892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:38.279129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:38.282311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:38.282679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:38.284833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:38.289775 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:39.141437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1213/2435] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:38 2022] POST /identity/v3/users => generated 292 bytes in 1024 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:39.171612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:39.184813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550f475-69a1-41b6-b485-9024551b0f17 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:39.192692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:39.194980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:39.196742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:39.196742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:39.199981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:39.202961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:39.220589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1223/2436] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:39.225655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:39.246190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8357be-c06f-4ba5-83df-6bfee4cec2a6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:39.254820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5000d-65cf-4053-bb6d-1da75ecb631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:39.255635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5000d-65cf-4053-bb6d-1da75ecb631c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:39.255993 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5000d-65cf-4053-bb6d-1da75ecb631c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:39.256274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5000d-65cf-4053-bb6d-1da75ecb631c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:39.256965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5000d-65cf-4053-bb6d-1da75ecb631c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:39.258655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5000d-65cf-4053-bb6d-1da75ecb631c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:39.273481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1214/2437] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:39.279514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:39.302225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607abda-a8bc-4ec0-8e8d-a320717edf80 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:39.308755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:39.309682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:39.310780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:39.311188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] PATH_INFO: `/v3/projects/038c965496554083b6abb3e38f01af20/users/8b620626de6a4fc491a47cc3bcef7efb/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:39.339060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] RBAC: Authorizing `identity:create_grant(project_id=038c965496554083b6abb3e38f01af20, user_id=8b620626de6a4fc491a47cc3bcef7efb, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:39.342934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:39.555616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1224/2438] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:39 2022] PUT /identity/v3/projects/038c965496554083b6abb3e38f01af20/users/8b620626de6a4fc491a47cc3bcef7efb/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:39.568197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:39.568722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:39.569116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:39.574272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:39.576709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:39.656934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] MFA Rules not processed for user `8b620626de6a4fc491a47cc3bcef7efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:39.660608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:39.745790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:39.757220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e9874-6a1a-4ac1-a24d-0171b4e1908b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:39.768437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:39.769222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:39.769571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:39.769941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] PATH_INFO: `/v3/users/ff664c7ba095413daad6b3f1ebc49cc0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:39.772436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff664c7ba095413daad6b3f1ebc49cc0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:39.774934 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:39.838583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1215/2439] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:39.844488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:39.865646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500cae2-aed5-4b74-9786-2634833270bb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:39.875746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:39.876566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:39.876859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:39.877145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:39.877820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:39.880420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:40.044872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1216/2440] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:39 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:40.050152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:40.142211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff664c7ba095413daad6b3f1ebc49cc0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:40.198803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679b2c-3d8d-4024-879c-27469df1c07c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:40.208880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:40.209545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:40.221942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:40.221942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:40.221942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:40.221942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:40.245721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff664c7ba095413daad6b3f1ebc49cc0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:40.264345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff664c7ba095413daad6b3f1ebc49cc0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:40.266028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff664c7ba095413daad6b3f1ebc49cc0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:40.277275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff664c7ba095413daad6b3f1ebc49cc0'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:40.291041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1225/2441] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:39 2022] DELETE /identity/v3/users/ff664c7ba095413daad6b3f1ebc49cc0 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:40.358665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1217/2442] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:40 2022] POST /identity/v3/users => generated 298 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:40.362167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:40.366702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:40.488857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dea8a-06b9-435f-91c5-1f8931e07c8e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:40.499641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:40.500337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:40.500722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:40.501104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:40.502126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:40.504019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:40.526011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1226/2443] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:40.529435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:40.552546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9b1d4-0550-44b9-a7a6-fa0fdcd12546 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:40.563405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:40.564555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:40.565052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:40.565875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] PATH_INFO: `/v3/projects/91704cdbda334479ac0fb34ae3ed9ff7/users/618085ceca9245f4827b3ed41757fd37/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:40.613774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91704cdbda334479ac0fb34ae3ed9ff7, user_id=618085ceca9245f4827b3ed41757fd37, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:40.620185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:40.772311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1218/2444] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:40 2022] PUT /identity/v3/projects/91704cdbda334479ac0fb34ae3ed9ff7/users/618085ceca9245f4827b3ed41757fd37/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:40.778361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:40.796114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41329619-90de-42b6-808a-27ad5baae55c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:40.911484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:40.913925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:40.914841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:40.915319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:40.915875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:40.916424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:40.917810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:40.921711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:40.951856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1227/2445] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:40.954037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:40.971118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0b415-bb20-491d-b4c7-bab01ede0e8d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:40.986920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:40.988380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:40.988839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:40.989117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:40.991022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] PATH_INFO: `/v3/projects/eaba67e3a54f4191b9b924305a83e4eb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:40.994593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a534f-4c5c-4249-9f93-fcc94c68e8a7 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:40.995733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaba67e3a54f4191b9b924305a83e4eb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:41.003355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:41.004486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40256da0-1a0e-4d15-b4d9-a7a336fd645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:41.006213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40256da0-1a0e-4d15-b4d9-a7a336fd645d admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:41.006839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40256da0-1a0e-4d15-b4d9-a7a336fd645d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:41.007252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40256da0-1a0e-4d15-b4d9-a7a336fd645d admin admin] PATH_INFO: `/v3/projects/91704cdbda334479ac0fb34ae3ed9ff7/users/618085ceca9245f4827b3ed41757fd37/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:41.010799 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40256da0-1a0e-4d15-b4d9-a7a336fd645d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91704cdbda334479ac0fb34ae3ed9ff7, user_id=618085ceca9245f4827b3ed41757fd37, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:41.016868 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40256da0-1a0e-4d15-b4d9-a7a336fd645d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:41.043673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1228/2446] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:40 2022] PUT /identity/v3/projects/91704cdbda334479ac0fb34ae3ed9ff7/users/618085ceca9245f4827b3ed41757fd37/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:41.067816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:41.068109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:41.068388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:41.101680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] MFA Rules not processed for user `618085ceca9245f4827b3ed41757fd37`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:41.103031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:41.195987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1229/2447] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:41.215067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaba67e3a54f4191b9b924305a83e4eb', 'request_id': 'req-302778a1-36e3-445a-8e64-988070754d0e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:41.215634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:41.220479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:41.225449 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaba67e3a54f4191b9b924305a83e4eb', 'request_id': 'req-302778a1-36e3-445a-8e64-988070754d0e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:41.256594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaba67e3a54f4191b9b924305a83e4eb', 'request_id': 'req-302778a1-36e3-445a-8e64-988070754d0e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:41.400542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] The token cache is being invalidate because project eaba67e3a54f4191b9b924305a83e4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:41.400936 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:41.401599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-302778a1-36e3-445a-8e64-988070754d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:41.403934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1219/2448] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:40 2022] DELETE /identity/v3/projects/eaba67e3a54f4191b9b924305a83e4eb => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:41.405575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d51532-ee4c-4112-a68e-027c31f37b88 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:41.406095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d51532-ee4c-4112-a68e-027c31f37b88 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:41.406578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d51532-ee4c-4112-a68e-027c31f37b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:41.407396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf64f6-0803-47a8-bbae-116c4c9be85b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:41.441073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-20d51532-ee4c-4112-a68e-027c31f37b88 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:41.441876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-20d51532-ee4c-4112-a68e-027c31f37b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:41.554007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:41.557408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:41.558379 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:41.558694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:41.565033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:41.621131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1220/2449] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:41.635973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0db6c-53a0-4a00-ae56-d8f3bc67401a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:41.635973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0db6c-53a0-4a00-ae56-d8f3bc67401a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:41.636800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0db6c-53a0-4a00-ae56-d8f3bc67401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:41.707619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-76e0db6c-53a0-4a00-ae56-d8f3bc67401a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:41.711048 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-76e0db6c-53a0-4a00-ae56-d8f3bc67401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:41.724580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:41.725486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:41.737485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1230/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:41.738641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:41.740929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:41.956066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1221/2451] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:41.972479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:41.973388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:41.974241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.007331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ce51-90b7-4b59-9f7c-23757719abcf service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.025196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.025853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.026183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.026454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.037398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:42.046258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:42.047703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.049214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.061497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1222/2452] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:42.061497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.063856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:42.073276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1231/2453] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:13:42.081226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.095314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79184470-3866-4983-96a1-aa0f2c878b59 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.101382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.102314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.108028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.108028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.108028 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:42.108384 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71dfdb-b211-4581-8a2c-09df468e3355 service swift] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.117998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.119689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.120387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.121820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.124276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:13:42.124918 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.128677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.128925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:42.165463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1232/2454] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:13:42.167091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.194348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52c94e-2009-4d2d-9500-e3e12549d046 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.204317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.205597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.206399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.207255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.208630 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.208821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.210212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.213339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.239219 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1223/2455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:42.334641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1233/2456] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:42 2022] POST /identity/v3/projects => generated 363 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:42.339340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.357945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3ac1-5b7a-4059-afde-951b61c2b2bf service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.526050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.526480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.526748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.527027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.529604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.529796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:42.531070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.681541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1224/2457] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:42 2022] POST /identity/v3/users => generated 282 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:42.682959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.695137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00baea42-2ebf-4dd4-9f45-02fcbcb1b508 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.700486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.701353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.701640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.701992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.702660 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.704387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.721966 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1234/2458] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:42.725137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.739307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a0339-bffa-45b2-9d9f-9a1a4615d071 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.748267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f56993-e8eb-4921-a89e-d51245e9ce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.749075 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56993-e8eb-4921-a89e-d51245e9ce4f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.749365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56993-e8eb-4921-a89e-d51245e9ce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.749737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56993-e8eb-4921-a89e-d51245e9ce4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.750581 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f56993-e8eb-4921-a89e-d51245e9ce4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.752431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f56993-e8eb-4921-a89e-d51245e9ce4f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.766423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1225/2459] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:42.772077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:42.792740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abd659-046b-4abb-877f-358e854fe69f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:42.801876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:42.802888 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:42.803417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:42.804809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] PATH_INFO: `/v3/projects/de86d5fe7258432cbab7abb955fad47f/users/d032e687edec43bfb4643164a9f3586b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:42.837249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=de86d5fe7258432cbab7abb955fad47f, user_id=d032e687edec43bfb4643164a9f3586b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:42.840128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:42.993063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1235/2460] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:42 2022] PUT /identity/v3/projects/de86d5fe7258432cbab7abb955fad47f/users/d032e687edec43bfb4643164a9f3586b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:43.008838 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:43.009093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:43.009363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:43.058182 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] MFA Rules not processed for user `d032e687edec43bfb4643164a9f3586b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:43.060769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:43.167583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1226/2461] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:43.173853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:43.260687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005c71f-6c82-4fd0-8515-b54efa4e60ac admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:43.267619 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:43.268395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:43.268740 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:43.269107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:43.269822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:43.272324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:43.383415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1236/2462] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:43 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:43.391497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:43.423580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63dba4-2c7c-4498-b05c-5bf4854b4fad None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:43.550583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:43.551269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:43.551554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:43.551827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:43.553477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:43.555006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:43.654450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1227/2463] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:43 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:43.660939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:43.692371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab175-0eda-4667-8181-8da22b74e3b1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:43.698634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:43.699439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:43.699882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:43.700348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:43.701579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:43.703556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:43.728633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1237/2464] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:43.734677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:43.753653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28e33e-1daa-406e-91bd-618f29178da5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:43.761970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:43.763579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:43.764063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:43.764559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] PATH_INFO: `/v3/projects/3fd09e9dec8647e3974695c59ed5fbaf/users/cc6d0b26b58941ee890e9ebf5b9df039/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:43.793589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd09e9dec8647e3974695c59ed5fbaf, user_id=cc6d0b26b58941ee890e9ebf5b9df039, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:43.798423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:43.896685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1228/2465] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:43 2022] PUT /identity/v3/projects/3fd09e9dec8647e3974695c59ed5fbaf/users/cc6d0b26b58941ee890e9ebf5b9df039/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:43.902113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:43.919954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8eb06-3950-41d1-9ef5-7f199be76d92 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:44.009153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5afc7-f661-4d3e-969c-4588acfdf794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:44.010842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5afc7-f661-4d3e-969c-4588acfdf794 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:44.011508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5afc7-f661-4d3e-969c-4588acfdf794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:44.012228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5afc7-f661-4d3e-969c-4588acfdf794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:44.014401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5afc7-f661-4d3e-969c-4588acfdf794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:44.017977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5afc7-f661-4d3e-969c-4588acfdf794 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:44.035818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1238/2466] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:44.040509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:44.052910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeb1be-8a3e-4d95-b127-80404b53bd33 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:44.060084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:44.060798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:44.061150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:44.061656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] PATH_INFO: `/v3/projects/3fd09e9dec8647e3974695c59ed5fbaf/users/cc6d0b26b58941ee890e9ebf5b9df039/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:44.063869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd09e9dec8647e3974695c59ed5fbaf, user_id=cc6d0b26b58941ee890e9ebf5b9df039, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:44.066713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:44.079170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1229/2467] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:44 2022] PUT /identity/v3/projects/3fd09e9dec8647e3974695c59ed5fbaf/users/cc6d0b26b58941ee890e9ebf5b9df039/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:44.090345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:44.090664 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:44.091603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:44.120541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:44.130242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] MFA Rules not processed for user `cc6d0b26b58941ee890e9ebf5b9df039`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:44.131588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:44.250580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1239/2468] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:44.260999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:44.430360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2974a-897d-4c77-bd15-4c38352a1bf6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:44.439851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:44.441343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:44.441846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:44.442371 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:44.463023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:44.468640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:44.491549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1230/2469] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:44.625784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:44.699207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:45.171042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:45.649369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:45.651676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:45.727059 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:45.814198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d39ef-6e09-4ad6-b68f-a2b20e629fa3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:45.818472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a4010-a63c-4226-846c-9c34132e0354 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:45.821946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:45.822790 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:45.823201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:45.823544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:45.826084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:45.826715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:45.827302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:45.827759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:45.828304 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] PATH_INFO: `/v3/users/d032e687edec43bfb4643164a9f3586b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:45.829959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d032e687edec43bfb4643164a9f3586b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:45.832185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:45.842680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:45.844827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:45.857864 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1240/2470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:46.135843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd032e687edec43bfb4643164a9f3586b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.136856 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd032e687edec43bfb4643164a9f3586b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.157248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd032e687edec43bfb4643164a9f3586b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.183751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:46.186887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:46.208308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa21bf-ba8f-494a-8cfd-e59830c69ca9 service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:46.214425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:46.215168 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:46.215860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:46.216823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] PATH_INFO: `/v3/users/8b620626de6a4fc491a47cc3bcef7efb` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:46.218728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b620626de6a4fc491a47cc3bcef7efb)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:46.220528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:46.227663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:46.252379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd032e687edec43bfb4643164a9f3586b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.269742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b9e80bd7-e64f-43f6-bf77-054b6cad600b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd032e687edec43bfb4643164a9f3586b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.286032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1231/2471] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:45 2022] DELETE /identity/v3/users/d032e687edec43bfb4643164a9f3586b => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:46.288501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:46.288887 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:46.289239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:46.312305 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:46.313674 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:46.321880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:46.422349 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1232/2472] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:46.435501 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:46.437437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:46.483219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b620626de6a4fc491a47cc3bcef7efb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.515673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7305e02-b0d6-4382-b501-46c4215e456a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:46.519854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb24d7-35cb-4262-aedc-8a7f18b3b926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:46.520542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb24d7-35cb-4262-aedc-8a7f18b3b926 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:46.520825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb24d7-35cb-4262-aedc-8a7f18b3b926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:46.521106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb24d7-35cb-4262-aedc-8a7f18b3b926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:46.531495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb24d7-35cb-4262-aedc-8a7f18b3b926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:46.533061 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb24d7-35cb-4262-aedc-8a7f18b3b926 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:46.544627 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1233/2473] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:46.556256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b620626de6a4fc491a47cc3bcef7efb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.580839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b620626de6a4fc491a47cc3bcef7efb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.582216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b620626de6a4fc491a47cc3bcef7efb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.599997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b620626de6a4fc491a47cc3bcef7efb'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:46.612753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1241/2474] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:46 2022] DELETE /identity/v3/users/8b620626de6a4fc491a47cc3bcef7efb => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:46.620239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:46.620755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:46.621264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:46.645204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:46.646796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:46.718911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1234/2475] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:46.729466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:46.730882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:46.804839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ebdb-c2fa-4e7f-ad19-3b6fdb4a3153 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:46.810195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:46.810913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:46.811318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:46.811598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:46.826059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:46.827453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:46.844870 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1242/2476] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:47.260052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:47.263963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:47.307013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e2c46-ba60-473e-9da8-9d1fe0409846 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:47.307013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:47.307013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:47.307013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:47.308057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] PATH_INFO: `/v3/projects/de86d5fe7258432cbab7abb955fad47f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:47.308057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=de86d5fe7258432cbab7abb955fad47f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:47.309396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:47.550365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de86d5fe7258432cbab7abb955fad47f', 'request_id': 'req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:47.557203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de86d5fe7258432cbab7abb955fad47f', 'request_id': 'req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:47.565396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de86d5fe7258432cbab7abb955fad47f', 'request_id': 'req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:47.656355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:47.656355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:47.673027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0881d7-cb08-403d-89fe-506a90fabf80 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:47.743035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:47.743780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:47.744064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:47.744346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] PATH_INFO: `/v3/projects/038c965496554083b6abb3e38f01af20` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:47.745606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=038c965496554083b6abb3e38f01af20)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:47.748067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:47.764190 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] The token cache is being invalidate because project de86d5fe7258432cbab7abb955fad47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:47.764708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:47.765651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:47.767782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1235/2477] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:47 2022] DELETE /identity/v3/projects/de86d5fe7258432cbab7abb955fad47f => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:47.772994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:47.775198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:47.799116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1be090-647a-4e6e-9be0-41a8ab80a9ff admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:47.805101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:47.805764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:47.806080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:47.806361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] PATH_INFO: `/v3/users/cc6d0b26b58941ee890e9ebf5b9df039` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:47.807614 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6d0b26b58941ee890e9ebf5b9df039)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:47.810532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:47.928784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:47.953706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038c965496554083b6abb3e38f01af20', 'request_id': 'req-97d151af-e5db-4181-ad70-8917f74829c7'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:47.962523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038c965496554083b6abb3e38f01af20', 'request_id': 'req-97d151af-e5db-4181-ad70-8917f74829c7'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.179533 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038c965496554083b6abb3e38f01af20', 'request_id': 'req-97d151af-e5db-4181-ad70-8917f74829c7'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.197529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] The token cache is being invalidate because project 038c965496554083b6abb3e38f01af20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:48.198219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.199073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6d0b26b58941ee890e9ebf5b9df039'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.199377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.200668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6d0b26b58941ee890e9ebf5b9df039'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.205628 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1243/2478] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:47 2022] DELETE /identity/v3/projects/038c965496554083b6abb3e38f01af20 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:48.207309 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:48.208810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:48.219475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6d0b26b58941ee890e9ebf5b9df039'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.281653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d151af-e5db-4181-ad70-8917f74829c7 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:48.287064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:48.287818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:48.288212 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:48.288649 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] PATH_INFO: `/v3/users/618085ceca9245f4827b3ed41757fd37` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:48.290119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=618085ceca9245f4827b3ed41757fd37)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:48.292515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:48.339497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6d0b26b58941ee890e9ebf5b9df039'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.349807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6d0b26b58941ee890e9ebf5b9df039'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.357461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1236/2479] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:47 2022] DELETE /identity/v3/users/cc6d0b26b58941ee890e9ebf5b9df039 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:48.374212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:48.555529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618085ceca9245f4827b3ed41757fd37'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.658816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618085ceca9245f4827b3ed41757fd37'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.678342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618085ceca9245f4827b3ed41757fd37'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.679227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618085ceca9245f4827b3ed41757fd37'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.688748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618085ceca9245f4827b3ed41757fd37'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:48.696614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1244/2480] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:48 2022] DELETE /identity/v3/users/618085ceca9245f4827b3ed41757fd37 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:48.999695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:49.007834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:49.133833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad6106-ce63-4cf1-978d-1b5f6e822956 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:49.225752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:49.226562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.228058 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.228343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] PATH_INFO: `/v3/projects/3fd09e9dec8647e3974695c59ed5fbaf` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.230326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd09e9dec8647e3974695c59ed5fbaf)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:49.233643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:49.336952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:49.338563 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:49.363755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d08e9-b7b3-42d2-851f-0aac5e5469f0 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:49.375336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:49.375336 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.376032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.376537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] PATH_INFO: `/v3/projects/91704cdbda334479ac0fb34ae3ed9ff7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.378460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91704cdbda334479ac0fb34ae3ed9ff7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:49.382376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:49.391720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd09e9dec8647e3974695c59ed5fbaf', 'request_id': 'req-aab3b0e8-77c7-49e1-876a-a465bc608ce7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.400109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd09e9dec8647e3974695c59ed5fbaf', 'request_id': 'req-aab3b0e8-77c7-49e1-876a-a465bc608ce7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.408170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd09e9dec8647e3974695c59ed5fbaf', 'request_id': 'req-aab3b0e8-77c7-49e1-876a-a465bc608ce7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.503015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] The token cache is being invalidate because project 3fd09e9dec8647e3974695c59ed5fbaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:49.503347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.504312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-aab3b0e8-77c7-49e1-876a-a465bc608ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.507442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1237/2481] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:48 2022] DELETE /identity/v3/projects/3fd09e9dec8647e3974695c59ed5fbaf => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:49.515041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc116-c680-450f-b7c5-b22b8e5862de None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.515643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc116-c680-450f-b7c5-b22b8e5862de None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.516079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc116-c680-450f-b7c5-b22b8e5862de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.544817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-216fc116-c680-450f-b7c5-b22b8e5862de None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:49.545527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91704cdbda334479ac0fb34ae3ed9ff7', 'request_id': 'req-4684e9d8-c38c-42d9-bb0b-2dea697d316e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.546672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-216fc116-c680-450f-b7c5-b22b8e5862de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:49.550508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:49.553558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91704cdbda334479ac0fb34ae3ed9ff7', 'request_id': 'req-4684e9d8-c38c-42d9-bb0b-2dea697d316e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.629006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91704cdbda334479ac0fb34ae3ed9ff7', 'request_id': 'req-4684e9d8-c38c-42d9-bb0b-2dea697d316e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.651963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] The token cache is being invalidate because project 91704cdbda334479ac0fb34ae3ed9ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:49.664300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1238/2482] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:49.665173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.665173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4684e9d8-c38c-42d9-bb0b-2dea697d316e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:49.665173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1245/2483] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:49 2022] DELETE /identity/v3/projects/91704cdbda334479ac0fb34ae3ed9ff7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:49.665889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1411e4e-0e2d-4363-a6ae-076c9e1d6b38 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.665889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1411e4e-0e2d-4363-a6ae-076c9e1d6b38 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.665889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1411e4e-0e2d-4363-a6ae-076c9e1d6b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.669173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.669575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.669996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.692115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:49.693498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:49.695402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f1411e4e-0e2d-4363-a6ae-076c9e1d6b38 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:49.697656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f1411e4e-0e2d-4363-a6ae-076c9e1d6b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:49.699106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:49.711373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1246/2484] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:49.712741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:49.721482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1239/2485] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:49.725432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104398-3100-4a10-80fc-0eb9623c99d5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:49.730876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:49.732383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.733126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.733403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.734293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:13:49.734519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:49.736258 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:49.742798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.743457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.744082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.755208 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1247/2486] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:13:49.760758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:49.779702 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925962e-715b-484e-8e92-e27f91d3d5c9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:49.785657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:49.788521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:49.788796 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:49.789209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.789679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.790201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.791556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:49.795364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:49.810464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1240/2487] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:49.815096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:49.829867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f934f6-8b1d-40aa-ae65-efd90c7d0075 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:49.836650 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:49.837286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.837573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.837852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.838552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:13:49.838859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:49.840649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:49.855310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1241/2488] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:13:49.861585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:49.875290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20681d-d578-4e4a-b41b-ebc26383aaa6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:49.881437 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:49.882296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:49.882677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:49.883252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:49.884133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:49.886569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:49.941269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1248/2489] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/projects => generated 391 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:49.946423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.036733 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1242/2490] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:50.039026 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77059b6e-386c-4c64-9e92-7be8427d8437 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.042670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.143970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc8856-4d3f-4c87-bc71-9c79536d136c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.150346 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.150994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.151400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.151722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.152995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.155390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.167499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.168516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.170077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.170611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.171828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.174300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.332439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1243/2491] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:50 2022] POST /identity/v3/users => generated 290 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:50.338396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.351696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637f14-9ded-421c-bb06-692f362758c6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.358968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:50.360322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.361729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.362319 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.362825 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.364107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.367285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.371327 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1249/2492] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:49 2022] POST /identity/v3/users => generated 296 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:50.376251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.387361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b4041-b1c5-4e19-b453-cbd4823c6b83 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.389043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1244/2493] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:13:50.392805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.399058 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.400127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.400657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.401146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.402392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.406549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.407366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e8fa-4cce-473b-a50e-3920d10bf913 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.414946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.415832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.416297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.416687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.417466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.419922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.431858 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1250/2494] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:13:50.432987 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1245/2495] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:13:50.434142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.445067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.446389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:50.454185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099f7eb-887e-468b-a3a6-b877aa906670 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.457518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970dbe2-3255-46a3-8f75-5d0d8496c394 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.463064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.464013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1394726-4f10-4ac5-a387-5c58761d163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.464144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.464593 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.465077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.465193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1394726-4f10-4ac5-a387-5c58761d163a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.466422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.466635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1394726-4f10-4ac5-a387-5c58761d163a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.466854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1394726-4f10-4ac5-a387-5c58761d163a admin admin] PATH_INFO: `/v3/projects/d25991f35d14479fb889e577f81c2c36/users/2336e77173a2413e861879ad2682019b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.469642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.486537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1394726-4f10-4ac5-a387-5c58761d163a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25991f35d14479fb889e577f81c2c36, user_id=2336e77173a2413e861879ad2682019b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.491520 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1394726-4f10-4ac5-a387-5c58761d163a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.494341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1251/2496] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:13:50.496773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.516112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcb97f-655a-4b7a-a4a4-a86b668e7ab7 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.524582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeafe2-7e54-4674-b3c2-34d1921e89b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.525806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeafe2-7e54-4674-b3c2-34d1921e89b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.526371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeafe2-7e54-4674-b3c2-34d1921e89b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.526863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeafe2-7e54-4674-b3c2-34d1921e89b3 admin admin] PATH_INFO: `/v3/projects/136a046b078f4b3ea302ea7802cf628f/users/3c3bd07a2ad646b2822f049c77e3f7bc/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.554640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeafe2-7e54-4674-b3c2-34d1921e89b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=136a046b078f4b3ea302ea7802cf628f, user_id=3c3bd07a2ad646b2822f049c77e3f7bc, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.560693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeafe2-7e54-4674-b3c2-34d1921e89b3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.615005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1246/2497] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:50 2022] PUT /identity/v3/projects/d25991f35d14479fb889e577f81c2c36/users/2336e77173a2413e861879ad2682019b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:50.635447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.635745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.636056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.648487 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1252/2498] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:50 2022] PUT /identity/v3/projects/136a046b078f4b3ea302ea7802cf628f/users/3c3bd07a2ad646b2822f049c77e3f7bc/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:50.660196 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.660526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.660730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.675754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] MFA Rules not processed for user `2336e77173a2413e861879ad2682019b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:50.677006 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:50.694727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] MFA Rules not processed for user `3c3bd07a2ad646b2822f049c77e3f7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:50.696109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:50.761499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1247/2499] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:50.766747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.802333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1253/2500] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:50.807226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:50.844933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75d650-3ad9-46a9-87bc-17dd3073f014 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.850443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.851209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.851499 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.851782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.852565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.854673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:50.903083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02223310-70bb-42bd-9d7a-f7a1b1ae3d47 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:50.908958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:50.909737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:50.910215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:50.910589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:50.911370 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:50.913546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.061723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:51.080355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1248/2501] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:50 2022] POST /identity/v3/projects => generated 377 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:51.084547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.130143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1254/2502] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:50 2022] POST /identity/v3/projects => generated 393 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:51.134887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.182944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedf1c9-a42d-4265-986e-65c62e88eea1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.251614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:51.294785 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.295545 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.295892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.296184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.297047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.299265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.321001 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e0eb4-a84d-4029-a4fe-f40434a74897 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.329757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.331180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.331319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.331997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.333189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.336967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.454947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1249/2503] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:51 2022] POST /identity/v3/users => generated 289 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:51.457991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.478103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2ac3d-94d3-427e-b3f9-0ee98dfaeef7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.484901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.485709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.486148 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1255/2504] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:51 2022] POST /identity/v3/users => generated 303 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:51.486629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.486629 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.487358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.489139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.490142 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.503573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1250/2505] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:51.505342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f5e1d-70ea-46b4-88c6-435ca73e9c13 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.507384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.516073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.517123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.517986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.518536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.520457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.522693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0297f6-2a0f-4cc2-b157-4ae3cad895e9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.522853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.530711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.531662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.532013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.532369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.533766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.536348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.554623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1256/2506] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:51.560331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1251/2507] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:51.560331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.560560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.571700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf09201-fd69-45d0-8f64-10b911f18808 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.571926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be260f2a-946c-4c74-9d4b-4f4805102f63 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.578779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212a90a-4ce1-4fe0-bf60-8152dbd85c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.579008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.579661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.579780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a90a-4ce1-4fe0-bf60-8152dbd85c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.579952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a90a-4ce1-4fe0-bf60-8152dbd85c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.580070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.580225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a90a-4ce1-4fe0-bf60-8152dbd85c49 admin admin] PATH_INFO: `/v3/projects/a35adb447f014e52a1a237a51a62d4df/users/2858814a061d4a8e9d8226aafe8e76ed/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.580376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] PATH_INFO: `/v3/projects/75f968b364344b3fbf86162ab194ce67/users/28669168d96d44f3a93f6f6ffe45f83c/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.601744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f968b364344b3fbf86162ab194ce67, user_id=28669168d96d44f3a93f6f6ffe45f83c, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.603711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a90a-4ce1-4fe0-bf60-8152dbd85c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35adb447f014e52a1a237a51a62d4df, user_id=2858814a061d4a8e9d8226aafe8e76ed, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.604900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.609419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a90a-4ce1-4fe0-bf60-8152dbd85c49 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.739894 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1257/2508] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:51 2022] PUT /identity/v3/projects/75f968b364344b3fbf86162ab194ce67/users/28669168d96d44f3a93f6f6ffe45f83c/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:51.744373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.768859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1252/2509] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:51 2022] PUT /identity/v3/projects/a35adb447f014e52a1a237a51a62d4df/users/2858814a061d4a8e9d8226aafe8e76ed/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:51.771198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.771767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.772376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.811517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] MFA Rules not processed for user `2858814a061d4a8e9d8226aafe8e76ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:51.813818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:51.874696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe1f24-0d22-4545-9a64-3b1d6cfc3cb3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:51.882117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:51.926106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1253/2510] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:51.931665 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:51.966696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:51.967625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:51.967910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:51.987636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:51.987636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:51.987636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:51.987636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1258/2511] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:51.990798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.004257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877dd3eb-b31e-4d62-a8fa-e3fa62aea807 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.009644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0e36d-9156-417f-8b60-8331ed9df9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.010365 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0e36d-9156-417f-8b60-8331ed9df9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.010646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0e36d-9156-417f-8b60-8331ed9df9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.010922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0e36d-9156-417f-8b60-8331ed9df9f6 admin admin] PATH_INFO: `/v3/projects/75f968b364344b3fbf86162ab194ce67/users/28669168d96d44f3a93f6f6ffe45f83c/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.012956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0e36d-9156-417f-8b60-8331ed9df9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f968b364344b3fbf86162ab194ce67, user_id=28669168d96d44f3a93f6f6ffe45f83c, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.016049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0e36d-9156-417f-8b60-8331ed9df9f6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.029725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1259/2512] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:51 2022] PUT /identity/v3/projects/75f968b364344b3fbf86162ab194ce67/users/28669168d96d44f3a93f6f6ffe45f83c/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:52.042096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.042476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.042808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.055662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a734d8-5d55-4824-9087-38a9af6dc6e9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.079067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] MFA Rules not processed for user `28669168d96d44f3a93f6f6ffe45f83c`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:52.080923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:52.108746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:52.148163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.149256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.149756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.150354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.151145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.155566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.184623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1260/2513] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:52.196625 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.198264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:52.281254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978425cc-e1e6-43e6-9692-5f13b77da712 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.285924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.286603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.286885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.287578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.299675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1254/2514] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:13:51 2022] POST /identity/v3/projects => generated 377 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:13:52.301217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.303511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.305804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.322731 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1261/2515] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:52.394750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.397032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:52.414973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995f2dd-b132-493c-a720-614c9c611b77 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.423524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd910cd0-a4ac-40c8-b204-ffb55f84c3ed service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.425640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.426840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.427419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.428013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.429312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.433269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.494010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.494886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.494886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.495272 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.560637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:52.572986 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1255/2516] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:13:52 2022] POST /identity/v3/users => generated 295 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:13:52.574767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.611849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52661d7b-54bf-4f80-834d-6577ad5c418e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.623318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.624877 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.627759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.629005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.629508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.632821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.632821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.635325 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.636886 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1262/2517] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:52.671615 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1256/2518] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:52.678642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.699070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ba25b-32ed-46c1-b0f2-7f98c635fba9 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:52.709412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:52.710317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:52.710677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:52.711025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] PATH_INFO: `/v3/projects/978d65d5f3c64741ba7f4bd4c562a9b2/users/de249f1d33cb44c6a56eb3bfcd827c28/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:52.723136 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:52.737580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=978d65d5f3c64741ba7f4bd4c562a9b2, user_id=de249f1d33cb44c6a56eb3bfcd827c28, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:52.740800 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:52.912494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1263/2519] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:52 2022] PUT /identity/v3/projects/978d65d5f3c64741ba7f4bd4c562a9b2/users/de249f1d33cb44c6a56eb3bfcd827c28/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:52.914549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:52.944073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a9252-b6c7-4a43-ac6c-b9e6e1d88a74 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:53.062100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951f4f5-b8f0-4e35-b524-f44056a6a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:53.063121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f4f5-b8f0-4e35-b524-f44056a6a644 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:53.063608 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f4f5-b8f0-4e35-b524-f44056a6a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:53.063836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f4f5-b8f0-4e35-b524-f44056a6a644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:53.064550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951f4f5-b8f0-4e35-b524-f44056a6a644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:53.066257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951f4f5-b8f0-4e35-b524-f44056a6a644 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:53.080014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1257/2520] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:13:53.086552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:53.119125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0638-8fa2-4652-9f61-500bff33be41 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:53.133543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:53.134280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:53.134568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:53.134851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] PATH_INFO: `/v3/projects/978d65d5f3c64741ba7f4bd4c562a9b2/users/de249f1d33cb44c6a56eb3bfcd827c28/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:53.139899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=978d65d5f3c64741ba7f4bd4c562a9b2, user_id=de249f1d33cb44c6a56eb3bfcd827c28, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:53.142852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:53.158530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1264/2521] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:13:53 2022] PUT /identity/v3/projects/978d65d5f3c64741ba7f4bd4c562a9b2/users/de249f1d33cb44c6a56eb3bfcd827c28/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:53.170718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:53.171433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:53.171886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:53.245335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] MFA Rules not processed for user `de249f1d33cb44c6a56eb3bfcd827c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:53.245335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:53.431732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1258/2522] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:13:53.451462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:53.573038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793d057-7f62-4ec9-9ef1-a847cf4e2d54 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:53.580177 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:53.581095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:53.581836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:53.582389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:53.602302 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:53.605023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:53.662093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1265/2523] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:53.841073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:53.875600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2c320-13b6-4fb1-84bd-409330a13404 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:53.911497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:53.918472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:53.918472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:53.918472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:54.085029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:54.089500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:54.107798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1259/2524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:57.010752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:57.012443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:57.095651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec395f-ecc1-4696-8068-88f28c911049 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:57.100006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:57.100729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:57.101080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:57.101420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:57.103403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:57.195527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:57.197005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:57.209840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1266/2525] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:13:58.639920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:58.652276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:58.735044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f248239-a3b8-4503-982d-476ad5b79abc service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:58.740185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:58.740764 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:58.741051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:58.741335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] PATH_INFO: `/v3/users/3c3bd07a2ad646b2822f049c77e3f7bc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:58.742594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3bd07a2ad646b2822f049c77e3f7bc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:58.744593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:59.158712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:59.172641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3bd07a2ad646b2822f049c77e3f7bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.173803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3bd07a2ad646b2822f049c77e3f7bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.176489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e405803-3ce7-40cc-9769-e5751aa4f3aa service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:59.185575 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:59.186710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:59.187205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:59.187695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] PATH_INFO: `/v3/users/2336e77173a2413e861879ad2682019b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:59.190499 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2336e77173a2413e861879ad2682019b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:59.194038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:59.209039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3bd07a2ad646b2822f049c77e3f7bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.238828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:59.308635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3bd07a2ad646b2822f049c77e3f7bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.319808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a191f017-7f29-4bd7-bdbe-3959453a3965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3bd07a2ad646b2822f049c77e3f7bc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.330165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1260/2526] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:58 2022] DELETE /identity/v3/users/3c3bd07a2ad646b2822f049c77e3f7bc => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:59.335541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:59.335803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:59.335971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:59.358945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:59.360233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:59.439483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1261/2527] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:59.452299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:59.454163 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:59.503195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2336e77173a2413e861879ad2682019b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.508487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:59.559614 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a35fd-9c7d-46c3-8f31-e92aba09be2a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:59.565597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c94a92-7826-4af2-bac4-5894af9c451c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:59.566620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c94a92-7826-4af2-bac4-5894af9c451c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:59.567051 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c94a92-7826-4af2-bac4-5894af9c451c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:59.567474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c94a92-7826-4af2-bac4-5894af9c451c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:59.584985 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c94a92-7826-4af2-bac4-5894af9c451c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:59.587267 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c94a92-7826-4af2-bac4-5894af9c451c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:59.600643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1262/2528] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:59.603910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2336e77173a2413e861879ad2682019b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.629361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2336e77173a2413e861879ad2682019b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.630574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2336e77173a2413e861879ad2682019b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.645758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2336e77173a2413e861879ad2682019b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:59.660465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1267/2529] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:59 2022] DELETE /identity/v3/users/2336e77173a2413e861879ad2682019b => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:13:59.666890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:59.667738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:59.667738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:59.699797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:13:59.702023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:59.805810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1263/2530] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:13:59.820853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:59.823103 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:59.932180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59a32-d0df-43fb-995c-2cc6ee228c26 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:59.939761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:59.941442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:59.942045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:59.942641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:59.962101 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:59.966106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:59.992315 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1268/2531] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:00.341839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:00.344717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:00.370550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754ad71-4ad6-4c7f-8883-5610f8652570 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:00.394226 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:00.395494 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:00.395960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:00.396657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] PATH_INFO: `/v3/projects/136a046b078f4b3ea302ea7802cf628f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:00.398358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=136a046b078f4b3ea302ea7802cf628f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:00.403460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:00.594227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136a046b078f4b3ea302ea7802cf628f', 'request_id': 'req-4c969c90-4346-4c5f-908a-d1fc231add0d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:00.614334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136a046b078f4b3ea302ea7802cf628f', 'request_id': 'req-4c969c90-4346-4c5f-908a-d1fc231add0d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:00.635155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136a046b078f4b3ea302ea7802cf628f', 'request_id': 'req-4c969c90-4346-4c5f-908a-d1fc231add0d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:00.744477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] The token cache is being invalidate because project 136a046b078f4b3ea302ea7802cf628f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:00.746047 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:00.750787 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:00.759971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1264/2532] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:00 2022] DELETE /identity/v3/projects/136a046b078f4b3ea302ea7802cf628f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:00.761924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:00.782060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe1458-8d3e-4b78-9658-b3c0bc79bc3c service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:00.883050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:00.886134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:00.891954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:00.892665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:00.892950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:00.893286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] PATH_INFO: `/v3/users/28669168d96d44f3a93f6f6ffe45f83c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:00.895112 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28669168d96d44f3a93f6f6ffe45f83c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:00.897330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:00.921711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c969c90-4346-4c5f-908a-d1fc231add0d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:00.949286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:00.955200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:00.957700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:00.959416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] PATH_INFO: `/v3/projects/d25991f35d14479fb889e577f81c2c36` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:00.969285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25991f35d14479fb889e577f81c2c36)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:00.976682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:01.259857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:01.264766 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28669168d96d44f3a93f6f6ffe45f83c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.273560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25991f35d14479fb889e577f81c2c36', 'request_id': 'req-6d387996-a9be-4fac-b787-587f873de689'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.276001 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:01.285957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25991f35d14479fb889e577f81c2c36', 'request_id': 'req-6d387996-a9be-4fac-b787-587f873de689'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.297031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25991f35d14479fb889e577f81c2c36', 'request_id': 'req-6d387996-a9be-4fac-b787-587f873de689'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.363362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28669168d96d44f3a93f6f6ffe45f83c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.387738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28669168d96d44f3a93f6f6ffe45f83c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.388925 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28669168d96d44f3a93f6f6ffe45f83c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.401861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28669168d96d44f3a93f6f6ffe45f83c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.409409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1269/2533] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:00 2022] DELETE /identity/v3/users/28669168d96d44f3a93f6f6ffe45f83c => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:01.482433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] The token cache is being invalidate because project d25991f35d14479fb889e577f81c2c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:01.483715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.484842 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:01.488171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1265/2534] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:00 2022] DELETE /identity/v3/projects/d25991f35d14479fb889e577f81c2c36 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:01.493163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:01.494559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:01.617431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529bcee-12c3-4f64-a9b2-cb4f8b8b89d9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:01.626387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:01.627524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:01.628222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:01.628734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] PATH_INFO: `/v3/users/2858814a061d4a8e9d8226aafe8e76ed` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:01.631020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2858814a061d4a8e9d8226aafe8e76ed)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:01.635048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:01.838059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2858814a061d4a8e9d8226aafe8e76ed'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.039588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2858814a061d4a8e9d8226aafe8e76ed'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.055729 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2858814a061d4a8e9d8226aafe8e76ed'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.056528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2858814a061d4a8e9d8226aafe8e76ed'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.065418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2858814a061d4a8e9d8226aafe8e76ed'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.072876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1270/2535] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:01 2022] DELETE /identity/v3/users/2858814a061d4a8e9d8226aafe8e76ed => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:02.099097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:02.104172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:02.232967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d387996-a9be-4fac-b787-587f873de689 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:02.315713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:02.316481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:02.316897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:02.317231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] PATH_INFO: `/v3/projects/75f968b364344b3fbf86162ab194ce67` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:02.319158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f968b364344b3fbf86162ab194ce67)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:02.322798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:02.482315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f968b364344b3fbf86162ab194ce67', 'request_id': 'req-6165844a-7fb6-40ee-a493-764cc50a1c7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.496428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f968b364344b3fbf86162ab194ce67', 'request_id': 'req-6165844a-7fb6-40ee-a493-764cc50a1c7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.506754 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f968b364344b3fbf86162ab194ce67', 'request_id': 'req-6165844a-7fb6-40ee-a493-764cc50a1c7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.577781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:02.581280 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:02.609296 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9565ad-7a1a-4d5b-8c27-dbbe721e2c08 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:02.627230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] The token cache is being invalidate because project 75f968b364344b3fbf86162ab194ce67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:02.628043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.629759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6165844a-7fb6-40ee-a493-764cc50a1c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.631215 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1266/2536] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:02 2022] DELETE /identity/v3/projects/75f968b364344b3fbf86162ab194ce67 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:02.641565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10679d-e124-4cd0-ad2b-609983a2278b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:02.642009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10679d-e124-4cd0-ad2b-609983a2278b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:02.642358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10679d-e124-4cd0-ad2b-609983a2278b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:02.675977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:02.679102 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fa10679d-e124-4cd0-ad2b-609983a2278b None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:02.681473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fa10679d-e124-4cd0-ad2b-609983a2278b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:02.694897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:02.695732 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:02.696023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:02.696307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] PATH_INFO: `/v3/projects/a35adb447f014e52a1a237a51a62d4df` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:02.697509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:02.697711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35adb447f014e52a1a237a51a62d4df)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:02.700434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:02.768882 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1267/2537] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:02.779302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:02.779649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:02.779975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:02.802154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:02.803415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:02.812762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1268/2538] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:02.817996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:02.830609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e056e-1b90-45c5-aaa4-a001c4dfd733 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:02.836239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:02.837319 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:02.837630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:02.837955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:02.838604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:14:02.838907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:02.840561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:02.855037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1269/2539] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:14:02.862100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:02.876959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976e8d-e726-4138-a473-356b38ccb5e3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:02.886184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:02.887932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:02.888744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:02.889550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:02.890790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:02.895022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:02.898992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35adb447f014e52a1a237a51a62d4df', 'request_id': 'req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.908254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35adb447f014e52a1a237a51a62d4df', 'request_id': 'req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:02.984565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35adb447f014e52a1a237a51a62d4df', 'request_id': 'req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.002811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] The token cache is being invalidate because project a35adb447f014e52a1a237a51a62d4df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:03.003343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.004453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.007257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1271/2540] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:02 2022] DELETE /identity/v3/projects/a35adb447f014e52a1a237a51a62d4df => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:03.013056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:03.015908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:03.039238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1270/2541] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:02 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:03.044322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:03.046330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:03.127881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa18f66-ef05-4314-8abc-8adb46fc1f25 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:03.134084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:03.134721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.135033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.135324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.136066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:03.138971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:03.147355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc14d0-2f11-4010-95f5-5cea06a63fd7 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:03.158141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:03.160586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.160586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.160586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] PATH_INFO: `/v3/users/de249f1d33cb44c6a56eb3bfcd827c28` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.162689 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] RBAC: Authorizing `identity:delete_user(user_id=de249f1d33cb44c6a56eb3bfcd827c28)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:03.166754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:03.271968 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1271/2542] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:03 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:03.276419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:03.289187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37301a-731f-481c-9dc8-c8f6d09af9b2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:03.295714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:03.296535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.297020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.297477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.298599 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:03.300508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:03.312780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1272/2543] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:03.317720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:03.330132 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929471cb-bda1-46d5-a229-d3dbb856e713 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:03.335760 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:03.336410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.336701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.337000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.337741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:03.339408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:03.353786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1273/2544] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:03.358836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:03.369835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e46ea-748d-4a48-9cf2-9a9b5c978860 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:03.377766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929a646-7e8f-4b24-8b39-3343e7b5fb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:03.378984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929a646-7e8f-4b24-8b39-3343e7b5fb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.379491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929a646-7e8f-4b24-8b39-3343e7b5fb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.379990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929a646-7e8f-4b24-8b39-3343e7b5fb10 admin admin] PATH_INFO: `/v3/projects/a301540e1c7d4fa2aaacbb4519b901c2/users/aacc0f275dad48919c7b1d8a33b6f148/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.409068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929a646-7e8f-4b24-8b39-3343e7b5fb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a301540e1c7d4fa2aaacbb4519b901c2, user_id=aacc0f275dad48919c7b1d8a33b6f148, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:03.414718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929a646-7e8f-4b24-8b39-3343e7b5fb10 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:03.438175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de249f1d33cb44c6a56eb3bfcd827c28'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.585105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1274/2545] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:03 2022] PUT /identity/v3/projects/a301540e1c7d4fa2aaacbb4519b901c2/users/aacc0f275dad48919c7b1d8a33b6f148/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:03.598159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.598556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.599077 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.623059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de249f1d33cb44c6a56eb3bfcd827c28'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.633818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] MFA Rules not processed for user `aacc0f275dad48919c7b1d8a33b6f148`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:03.635134 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:03.641833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:03.642495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de249f1d33cb44c6a56eb3bfcd827c28'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.643872 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de249f1d33cb44c6a56eb3bfcd827c28'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.646059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:03.656036 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de249f1d33cb44c6a56eb3bfcd827c28'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:03.664921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1272/2546] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:03 2022] DELETE /identity/v3/users/de249f1d33cb44c6a56eb3bfcd827c28 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:03.717593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1275/2547] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:03.734071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:03.735668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:03.806965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaf872-c6e9-4950-8bf8-8eb4311269df admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:03.812028 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad6f9b-2876-4018-a2ee-e6cf83ec40f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:03.812725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad6f9b-2876-4018-a2ee-e6cf83ec40f5 service glance] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:03.813068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad6f9b-2876-4018-a2ee-e6cf83ec40f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:03.813483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad6f9b-2876-4018-a2ee-e6cf83ec40f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:03.902329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad6f9b-2876-4018-a2ee-e6cf83ec40f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:03.905290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad6f9b-2876-4018-a2ee-e6cf83ec40f5 service glance] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:03.915964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1273/2548] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:04.473503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:04.475116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:04.571621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ca0e-ea88-48ef-860a-60ed68b30281 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:04.670606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:04.670606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:04.670606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:04.670606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] PATH_INFO: `/v3/projects/978d65d5f3c64741ba7f4bd4c562a9b2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:04.671482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=978d65d5f3c64741ba7f4bd4c562a9b2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:04.673701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:04.876801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '978d65d5f3c64741ba7f4bd4c562a9b2', 'request_id': 'req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:04.884980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '978d65d5f3c64741ba7f4bd4c562a9b2', 'request_id': 'req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:04.887553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:04.892456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '978d65d5f3c64741ba7f4bd4c562a9b2', 'request_id': 'req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:05.056503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] The token cache is being invalidate because project 978d65d5f3c64741ba7f4bd4c562a9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:05.057374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:05.058757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-acf65c93-5ae2-4176-ab60-fb5cf93a88a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:05.062284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1276/2549] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:04 2022] DELETE /identity/v3/projects/978d65d5f3c64741ba7f4bd4c562a9b2 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:05.071600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.072230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.072652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.108180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:05.110352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:05.206610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1274/2550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:05.215373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.218118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.218118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.247392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:05.249288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:05.268343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1277/2551] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:05.270166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:05.282468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92feb9-63fc-4cd5-b296-dcd67597a031 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:05.288158 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:05.289023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.289451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.289933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.290760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:14:05.291188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:05.292962 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:05.308201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1275/2552] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:14:05.315519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:05.325422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241c1bb-63a5-41d6-b46d-8e2c905f0173 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:05.330945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:05.331840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.332124 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.332403 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.333070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:05.335145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:05.447280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1278/2553] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:05 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:05.452555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:05.464605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13175c87-5e30-4b83-be1c-a9dcace30413 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:05.521928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:05.573579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:05.574472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.581173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.581173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.581173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:05.581173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:05.722772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1276/2554] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:05 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:05.724015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:05.737143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5422417-cb45-4297-b8e6-c80104d9811d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:05.743108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:05.743918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.744271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.744730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.745571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:05.748654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:05.764101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1279/2555] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:05.773213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:05.786853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b85270-2fcc-4a76-b75d-48860b281745 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:05.792895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8d9bd-db54-4ecb-8531-0b026a661fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:05.793536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8d9bd-db54-4ecb-8531-0b026a661fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.794535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8d9bd-db54-4ecb-8531-0b026a661fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.799020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8d9bd-db54-4ecb-8531-0b026a661fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.799020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8d9bd-db54-4ecb-8531-0b026a661fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:05.799020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8d9bd-db54-4ecb-8531-0b026a661fd7 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:05.812773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1277/2556] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:05.818355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:05.832782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79169b-6e3d-4fb9-98b8-e2d8b4793519 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:05.838419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:05.847938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:05.847938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:05.847938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] PATH_INFO: `/v3/projects/ca6c076dc27141fbbcb9f3906ad62231/users/7c6d6ed26cf7486587491c39dcd1cd38/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:05.860148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6c076dc27141fbbcb9f3906ad62231, user_id=7c6d6ed26cf7486587491c39dcd1cd38, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:05.863369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:05.938997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1280/2557] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:05 2022] PUT /identity/v3/projects/ca6c076dc27141fbbcb9f3906ad62231/users/7c6d6ed26cf7486587491c39dcd1cd38/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:06.019564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:06.019860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:06.020119 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:06.054555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] MFA Rules not processed for user `7c6d6ed26cf7486587491c39dcd1cd38`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:06.055854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:06.141688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1278/2558] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:06.156609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:06.158865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:06.252826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ce159-9b4a-4d4b-aa6d-e23d0c59a815 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:06.256625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27e4e8-c1d5-46c0-8b81-8ca2f60a963a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:06.257258 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27e4e8-c1d5-46c0-8b81-8ca2f60a963a service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:06.257546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27e4e8-c1d5-46c0-8b81-8ca2f60a963a service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:06.257817 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27e4e8-c1d5-46c0-8b81-8ca2f60a963a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:06.268767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27e4e8-c1d5-46c0-8b81-8ca2f60a963a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:06.270335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27e4e8-c1d5-46c0-8b81-8ca2f60a963a service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:06.284513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1281/2559] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:16.137952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:16.276020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=57,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:17.196110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:17.271161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed91dd-035d-4a86-abf2-9794b3dbcb00 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:17.275941 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:17.276599 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:17.276887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:17.277170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] PATH_INFO: `/v3/users/7c6d6ed26cf7486587491c39dcd1cd38` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:17.279355 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6d6ed26cf7486587491c39dcd1cd38)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:17.281339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:17.520074 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6d6ed26cf7486587491c39dcd1cd38'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:17.607659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6d6ed26cf7486587491c39dcd1cd38'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:17.623411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6d6ed26cf7486587491c39dcd1cd38'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:17.624648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6d6ed26cf7486587491c39dcd1cd38'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:17.639193 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6d6ed26cf7486587491c39dcd1cd38'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:17.650584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1279/2560] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:17 2022] DELETE /identity/v3/users/7c6d6ed26cf7486587491c39dcd1cd38 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:17.656007 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:17.656574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:17.656574 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:17.677084 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:17.678630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:17.752589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1282/2561] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:17.763991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:17.766905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:17.845195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4af1a-0658-447b-a304-4f29d5ada870 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:17.849561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ef59b-bbbd-48a6-b7d4-80a63bbb1b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:17.850651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ef59b-bbbd-48a6-b7d4-80a63bbb1b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:17.851060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ef59b-bbbd-48a6-b7d4-80a63bbb1b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:17.852237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ef59b-bbbd-48a6-b7d4-80a63bbb1b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:17.868773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ef59b-bbbd-48a6-b7d4-80a63bbb1b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:17.872075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ef59b-bbbd-48a6-b7d4-80a63bbb1b9f service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:17.891171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1280/2562] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:18.467142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:18.469117 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:18.486776 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404a203-867d-4f72-8265-3d0b3b6001ab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:18.492630 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:18.493580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:18.494125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:18.494420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] PATH_INFO: `/v3/projects/ca6c076dc27141fbbcb9f3906ad62231` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:18.496621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6c076dc27141fbbcb9f3906ad62231)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:18.498389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:18.683381 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6c076dc27141fbbcb9f3906ad62231', 'request_id': 'req-de6c6311-3919-481f-930a-4b943aec5cc7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:18.690874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6c076dc27141fbbcb9f3906ad62231', 'request_id': 'req-de6c6311-3919-481f-930a-4b943aec5cc7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:18.698145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6c076dc27141fbbcb9f3906ad62231', 'request_id': 'req-de6c6311-3919-481f-930a-4b943aec5cc7'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:18.800052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] The token cache is being invalidate because project ca6c076dc27141fbbcb9f3906ad62231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:18.800347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:18.801079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-de6c6311-3919-481f-930a-4b943aec5cc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:18.802685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1283/2563] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:18 2022] DELETE /identity/v3/projects/ca6c076dc27141fbbcb9f3906ad62231 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:18.809059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:18.809059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:18.810935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:18.832677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:18.834190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:18.855811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:18.909146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1281/2564] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:18.921795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:18.922109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:18.922370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:18.945663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:18.947003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:18.956389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1284/2565] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:18.961392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:18.973423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a40c3-f02d-483a-a9c8-2ee2be2c5e26 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:18.978827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:18.979473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:18.979752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:18.980027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:18.981401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:14:18.982006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:18.983745 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:18.998044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1282/2566] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:14:19.003429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:19.014729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff36f21-87aa-4292-be6a-d88e5392d035 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:19.020701 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:19.021269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.021556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.021833 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.022529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:19.024487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:19.278481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1285/2567] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:19 2022] POST /identity/v3/projects => generated 377 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:19.283171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:19.295522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30f1f1-a45f-46da-976f-97b30ab65835 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:19.367741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:19.368331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.368618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.368899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.369640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:19.372601 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:19.476698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1283/2568] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:19 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:19.481280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:19.492360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cf4cc-28ce-4c16-a215-634eed2e3c0f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:19.498393 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:19.499055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.499342 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.499616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.500375 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:19.502101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:19.514188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1286/2569] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:19.519986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:19.531113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e087e2-68f8-4dcc-bd20-177ec7a6032e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:19.535989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafbd94-855c-4807-92f3-147036ce181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:19.536672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafbd94-855c-4807-92f3-147036ce181b admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.536993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafbd94-855c-4807-92f3-147036ce181b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.537268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafbd94-855c-4807-92f3-147036ce181b admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.537995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafbd94-855c-4807-92f3-147036ce181b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:19.539785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafbd94-855c-4807-92f3-147036ce181b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:19.553677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1284/2570] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:19.558127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:19.570232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca706ed4-ace6-4a8e-b0c0-9fcced590f3d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:19.576016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:19.576737 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.577024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.577307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] PATH_INFO: `/v3/projects/be1ba0ac49c04c8e818fcfb3a64cc6ad/users/48563fdd9b9a4f8faa1963a39bec1f5c/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.594376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1ba0ac49c04c8e818fcfb3a64cc6ad, user_id=48563fdd9b9a4f8faa1963a39bec1f5c, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:19.597254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:19.726256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1287/2571] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:19 2022] PUT /identity/v3/projects/be1ba0ac49c04c8e818fcfb3a64cc6ad/users/48563fdd9b9a4f8faa1963a39bec1f5c/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:19.743161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.743582 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.743875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.778669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] MFA Rules not processed for user `48563fdd9b9a4f8faa1963a39bec1f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:19.780786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:19.861723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1285/2572] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:19.866947 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:19.892925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:19.940104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df483e-fc43-465b-90ea-dd99abbdcf99 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:19.945106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:19.945806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:19.946160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:19.946720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:19.948042 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:19.950727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:20.095405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1288/2573] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:19 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:20.100844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:20.113457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabd569-6ada-4a14-866c-db3ec9991523 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:20.142520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:20.204032 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:20.204663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:20.204943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:20.205217 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:20.205859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:20.208129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:20.334198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1286/2574] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:20 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:20.338687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:20.349807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679c973-9e90-4831-b614-2682fe3f43fa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:20.354496 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:20.355133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:20.355919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:20.356209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:20.356862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:20.358539 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:20.370296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1289/2575] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:20.378396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:20.389600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966ee52-b03a-4efd-b0d9-da238679b8ca admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:20.394703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962e553-3b04-4b65-a7ca-6fce528d40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:20.395340 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962e553-3b04-4b65-a7ca-6fce528d40fc admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:20.395629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962e553-3b04-4b65-a7ca-6fce528d40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:20.395915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962e553-3b04-4b65-a7ca-6fce528d40fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:20.396578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962e553-3b04-4b65-a7ca-6fce528d40fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:20.398281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962e553-3b04-4b65-a7ca-6fce528d40fc admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:20.410843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1287/2576] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:20.416732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:20.428792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a61169-b9cf-4f5d-8bfd-a051e9ec8703 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:20.435602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:20.436408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:20.436759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:20.437087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] PATH_INFO: `/v3/projects/bab334d3b33441dea40436ba884261d0/users/e6d68623cb81416a9a30be8c06295d4f/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:20.455741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab334d3b33441dea40436ba884261d0, user_id=e6d68623cb81416a9a30be8c06295d4f, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:20.458795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:20.637561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1290/2577] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:20 2022] PUT /identity/v3/projects/bab334d3b33441dea40436ba884261d0/users/e6d68623cb81416a9a30be8c06295d4f/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:20.651571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:20.651571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:20.651571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:20.685242 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] MFA Rules not processed for user `e6d68623cb81416a9a30be8c06295d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:20.686540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:20.766420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1288/2578] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:20.771178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:20.849661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e7a97-5465-4adc-b753-c379b785fa44 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:20.854896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:20.856596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:20.856880 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:20.857209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:20.858033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:20.860108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:20.991904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1291/2579] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:20 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:20.996694 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:21.008206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c873667-6121-4465-8248-c2a1bd41fbe3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:21.075652 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:21.080854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.080854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.080854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.080854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:21.080854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:21.176515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1289/2580] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:20 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:21.181820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:21.195591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aae229-63c3-43cd-9d18-b6789b4d68ee admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:21.202703 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:21.203387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.203729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.204052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.204849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:21.207220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:21.222088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1292/2581] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:21.226756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:21.237673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d61a2-53c7-431b-bb15-ccaedeab3586 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:21.242844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:21.243589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.243913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.244234 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] PATH_INFO: `/v3/projects/61ae4648ab484d41b7b5cd4aea22f5d3/users/fe0ed31ca01945939ca7dbb337ef0e96/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.259221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:21.263876 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ae4648ab484d41b7b5cd4aea22f5d3, user_id=fe0ed31ca01945939ca7dbb337ef0e96, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:21.266825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:21.380826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1290/2582] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:21 2022] PUT /identity/v3/projects/61ae4648ab484d41b7b5cd4aea22f5d3/users/fe0ed31ca01945939ca7dbb337ef0e96/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:21.385376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:21.397678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bd87-dc44-4048-8202-4358784a9347 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:21.447351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:21.468311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fcdbe-0f5d-415f-8925-a74831205753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:21.468998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fcdbe-0f5d-415f-8925-a74831205753 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.469280 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fcdbe-0f5d-415f-8925-a74831205753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.469557 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fcdbe-0f5d-415f-8925-a74831205753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.470261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fcdbe-0f5d-415f-8925-a74831205753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:21.471876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fcdbe-0f5d-415f-8925-a74831205753 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:21.487488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1293/2583] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:21.492633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:21.512899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5490c-1326-42c1-b79e-4575e21c7044 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:21.521779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:21.523017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.523574 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.524080 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] PATH_INFO: `/v3/projects/61ae4648ab484d41b7b5cd4aea22f5d3/users/fe0ed31ca01945939ca7dbb337ef0e96/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.527605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ae4648ab484d41b7b5cd4aea22f5d3, user_id=fe0ed31ca01945939ca7dbb337ef0e96, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:21.532568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:21.549362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1291/2584] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:21 2022] PUT /identity/v3/projects/61ae4648ab484d41b7b5cd4aea22f5d3/users/fe0ed31ca01945939ca7dbb337ef0e96/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:21.560925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.561260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.561583 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.594848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] MFA Rules not processed for user `fe0ed31ca01945939ca7dbb337ef0e96`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:21.596161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:21.675741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1294/2585] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:21.688715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:21.690578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:21.759048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228d5d7-9673-4bb8-a410-5ca2e650f0eb admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:21.765430 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:21.766122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:21.766409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:21.766688 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:21.856108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:21.858813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:21.868320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1292/2586] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:24.131729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:24.137041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:24.145265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58324d0-b73f-489a-be99-e11c282f3369 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:24.149378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782490cb-2135-4f3a-a7af-67ead6869ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:24.150033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782490cb-2135-4f3a-a7af-67ead6869ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:24.150318 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782490cb-2135-4f3a-a7af-67ead6869ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:24.150596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782490cb-2135-4f3a-a7af-67ead6869ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:24.220505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782490cb-2135-4f3a-a7af-67ead6869ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:24.221988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782490cb-2135-4f3a-a7af-67ead6869ce1 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:24.233159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1295/2587] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:25.264240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:25.266041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:25.352117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41516c-2be9-42f8-98b1-228a5ddc012b service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:25.357439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:25.358135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:25.358418 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:25.358677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] PATH_INFO: `/v3/users/aacc0f275dad48919c7b1d8a33b6f148` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:25.359947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacc0f275dad48919c7b1d8a33b6f148)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:25.361929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:25.660231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:25.673190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacc0f275dad48919c7b1d8a33b6f148'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:25.790130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacc0f275dad48919c7b1d8a33b6f148'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:25.806738 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacc0f275dad48919c7b1d8a33b6f148'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:25.807665 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacc0f275dad48919c7b1d8a33b6f148'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:25.819906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacc0f275dad48919c7b1d8a33b6f148'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:25.835005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1293/2588] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:25 2022] DELETE /identity/v3/users/aacc0f275dad48919c7b1d8a33b6f148 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:25.842821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:25.843443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:25.843443 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:25.872493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:25.874085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:25.949923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1296/2589] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:25.961043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:25.962632 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:26.038220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633435-7e10-4797-9ab3-3e473393da11 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:26.042486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:26.043974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:26.044303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:26.044682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:26.056880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:26.060281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:26.071291 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1294/2590] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:26.279979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:26.281549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:26.288306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:26.366543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe36d5-9015-43e6-b46f-bedcd9e67127 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:26.372595 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:26.373461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:26.373808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:26.374196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:26.376486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:26.467257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:26.468783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7e3ec-e84f-43f8-8716-a2fe12e554b2 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:26.479859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1297/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:26.658705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:26.658705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:26.672826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001a3ff-d96c-40c5-9568-806619551d9a service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:26.677401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:26.678095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:26.678375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:26.678725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] PATH_INFO: `/v3/projects/a301540e1c7d4fa2aaacbb4519b901c2` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:26.679970 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a301540e1c7d4fa2aaacbb4519b901c2)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:26.681924 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:26.850246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a301540e1c7d4fa2aaacbb4519b901c2', 'request_id': 'req-26853e91-d521-4160-997c-7ed4ccdf5f23'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:26.858451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a301540e1c7d4fa2aaacbb4519b901c2', 'request_id': 'req-26853e91-d521-4160-997c-7ed4ccdf5f23'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:27.035657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a301540e1c7d4fa2aaacbb4519b901c2', 'request_id': 'req-26853e91-d521-4160-997c-7ed4ccdf5f23'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:27.049222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] The token cache is being invalidate because project a301540e1c7d4fa2aaacbb4519b901c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:27.049537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:27.050201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-26853e91-d521-4160-997c-7ed4ccdf5f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:27.051763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1295/2592] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:26 2022] DELETE /identity/v3/projects/a301540e1c7d4fa2aaacbb4519b901c2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:27.056823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.057389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.057389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.082204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:27.084372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:27.164816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1298/2593] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:27.173623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.173997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.174283 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.197376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:27.198684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:27.211293 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1296/2594] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:27.222515 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:27.234089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518cefe-6b3a-4021-9c31-99060920c158 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:27.239032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:27.239661 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.239943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.240212 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.240874 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:14:27.241180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:27.242830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:27.256052 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1299/2595] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:14:27.260767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:27.271578 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bc697-ce0d-47cc-afb9-0dbf0a7509a5 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:27.279883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:27.280569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.280913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.281421 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.281798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:27.283818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:27.401458 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1297/2596] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:27 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:27.406338 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:27.419800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43064a4a-5b9d-495f-b61b-2e61e356d6df admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:27.468020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:27.490658 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:27.491774 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.492116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.492446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.493217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:27.495473 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:27.660710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1300/2597] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:27 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:27.667241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:27.680751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc98832-77fb-4926-ab7a-fb231170ce87 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:27.688747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:27.692490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.692490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.692490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.692490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:27.692490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:27.694875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:27.703898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1298/2598] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:27.709380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:27.729529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f512e0-1bcb-4d7e-89ef-1bdfaec001e1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:27.738526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706ad87-ac96-4060-b1db-f66d7030652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:27.739594 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706ad87-ac96-4060-b1db-f66d7030652e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.740104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706ad87-ac96-4060-b1db-f66d7030652e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.740612 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706ad87-ac96-4060-b1db-f66d7030652e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.741801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706ad87-ac96-4060-b1db-f66d7030652e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:27.744979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706ad87-ac96-4060-b1db-f66d7030652e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:27.767232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1301/2599] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:27.772204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:27.786222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d3b80-f5d3-41b8-8cc6-cf9e6f94db6f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:27.792143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:27.793401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.793659 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.793940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] PATH_INFO: `/v3/projects/e0a37bc3a9a64bd6bb87348f829b2566/users/454e70d616544879b1c88796692dc8e3/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:27.816047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a37bc3a9a64bd6bb87348f829b2566, user_id=454e70d616544879b1c88796692dc8e3, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:27.818755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:27.964003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1299/2600] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:27 2022] PUT /identity/v3/projects/e0a37bc3a9a64bd6bb87348f829b2566/users/454e70d616544879b1c88796692dc8e3/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:27.982540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:27.982840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:27.983095 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:28.064493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] MFA Rules not processed for user `454e70d616544879b1c88796692dc8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:28.066980 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:28.166538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1302/2601] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:28.185587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:28.200095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:28.278739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e9a5a-f2a7-4086-9daa-7dd588bd51b4 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:28.283771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:28.284684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:28.285163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:28.285532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:28.298084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:28.299926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:28.315240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1300/2602] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:31.339085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:31.341543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:31.419418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25174298-e5e8-45b2-8a86-40f59c2f5079 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:31.424479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:31.425105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:31.425386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:31.425663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:31.427576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:31.516559 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:31.518532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4a154-2d98-4a73-b9cf-6eec6be1f389 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:31.532170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1303/2603] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:32.039693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:32.046216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:32.158290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5cd23-64e7-472e-b3e9-b7c9f1bf28a0 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:32.165285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:32.165547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:32.165676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:32.165978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] PATH_INFO: `/v3/users/48563fdd9b9a4f8faa1963a39bec1f5c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:32.167230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=48563fdd9b9a4f8faa1963a39bec1f5c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:32.170276 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:32.477803 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48563fdd9b9a4f8faa1963a39bec1f5c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:32.583697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48563fdd9b9a4f8faa1963a39bec1f5c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:32.614390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48563fdd9b9a4f8faa1963a39bec1f5c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:32.615656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48563fdd9b9a4f8faa1963a39bec1f5c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:32.627005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:32.630522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48563fdd9b9a4f8faa1963a39bec1f5c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:32.643373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1301/2604] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:32 2022] DELETE /identity/v3/users/48563fdd9b9a4f8faa1963a39bec1f5c => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:32.649096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:32.649977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:32.649977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:32.660734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:32.673543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:32.675572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:32.784617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1304/2605] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:32.798149 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:32.800339 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:32.907069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c5e24-1f54-4692-b0fe-adf88ab2d58c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:32.913088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:32.913564 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:32.913943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:32.914354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:32.929655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:32.931930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:32.946809 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1302/2606] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:33.634071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:33.637313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:33.666020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fad80-c600-4d64-b13a-bd667f31f872 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:33.681992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384750-e052-43ad-8234-51482703393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:33.681992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384750-e052-43ad-8234-51482703393a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:33.681992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384750-e052-43ad-8234-51482703393a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:33.681992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384750-e052-43ad-8234-51482703393a admin admin] PATH_INFO: `/v3/projects/be1ba0ac49c04c8e818fcfb3a64cc6ad` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:33.683869 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384750-e052-43ad-8234-51482703393a admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1ba0ac49c04c8e818fcfb3a64cc6ad)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:33.685128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384750-e052-43ad-8234-51482703393a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:33.908009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-24384750-e052-43ad-8234-51482703393a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1ba0ac49c04c8e818fcfb3a64cc6ad', 'request_id': 'req-24384750-e052-43ad-8234-51482703393a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:33.916086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-24384750-e052-43ad-8234-51482703393a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1ba0ac49c04c8e818fcfb3a64cc6ad', 'request_id': 'req-24384750-e052-43ad-8234-51482703393a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:33.923350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-24384750-e052-43ad-8234-51482703393a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1ba0ac49c04c8e818fcfb3a64cc6ad', 'request_id': 'req-24384750-e052-43ad-8234-51482703393a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.127973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-24384750-e052-43ad-8234-51482703393a admin admin] The token cache is being invalidate because project be1ba0ac49c04c8e818fcfb3a64cc6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:34.128273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-24384750-e052-43ad-8234-51482703393a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.129005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-24384750-e052-43ad-8234-51482703393a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.130636 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1305/2607] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:33 2022] DELETE /identity/v3/projects/be1ba0ac49c04c8e818fcfb3a64cc6ad => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:34.137928 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:34.157570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39161466-793a-4a9e-8dd8-a95daab81a85 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:34.237317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:34.238017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:34.238305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:34.238586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] PATH_INFO: `/v3/users/e6d68623cb81416a9a30be8c06295d4f` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:34.239834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d68623cb81416a9a30be8c06295d4f)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:34.242151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:34.522308 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d68623cb81416a9a30be8c06295d4f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.733328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d68623cb81416a9a30be8c06295d4f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.752705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d68623cb81416a9a30be8c06295d4f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.753556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d68623cb81416a9a30be8c06295d4f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.767751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d68623cb81416a9a30be8c06295d4f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:34.780146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1303/2608] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:34 2022] DELETE /identity/v3/users/e6d68623cb81416a9a30be8c06295d4f => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:35.513542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384750-e052-43ad-8234-51482703393a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:35.515944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-24384750-e052-43ad-8234-51482703393a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:35.540739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384750-e052-43ad-8234-51482703393a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:35.659165 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:35.660073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:35.660454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:35.660863 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] PATH_INFO: `/v3/projects/bab334d3b33441dea40436ba884261d0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:35.662471 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab334d3b33441dea40436ba884261d0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:35.665073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:35.859423 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:35.899519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab334d3b33441dea40436ba884261d0', 'request_id': 'req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:35.911958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab334d3b33441dea40436ba884261d0', 'request_id': 'req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:35.922498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab334d3b33441dea40436ba884261d0', 'request_id': 'req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.051302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] The token cache is being invalidate because project bab334d3b33441dea40436ba884261d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:36.051729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.052871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.055635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1306/2609] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:35 2022] DELETE /identity/v3/projects/bab334d3b33441dea40436ba884261d0 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:36.062480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:36.064852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:36.082467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270a827-c999-4fcc-a0c4-482d6e0ced70 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:36.119010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:36.181178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:36.182132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:36.182991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:36.182991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] PATH_INFO: `/v3/users/fe0ed31ca01945939ca7dbb337ef0e96` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:36.185333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0ed31ca01945939ca7dbb337ef0e96)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:36.187004 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:36.452722 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0ed31ca01945939ca7dbb337ef0e96'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.585332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0ed31ca01945939ca7dbb337ef0e96'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.615006 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0ed31ca01945939ca7dbb337ef0e96'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.616159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0ed31ca01945939ca7dbb337ef0e96'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.627020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0ed31ca01945939ca7dbb337ef0e96'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:36.636433 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1304/2610] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:36 2022] DELETE /identity/v3/users/fe0ed31ca01945939ca7dbb337ef0e96 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:37.283721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:37.288034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:37.368438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b71c-ee80-4ecf-8f8c-ce8f9cd57be3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:37.412723 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:37.415342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:37.507005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:37.508400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:37.508873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:37.509472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] PATH_INFO: `/v3/projects/61ae4648ab484d41b7b5cd4aea22f5d3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:37.511106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ae4648ab484d41b7b5cd4aea22f5d3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:37.514796 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:37.541471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19292562-2085-4a0b-8343-262f90e82226 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:37.547205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:37.548248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:37.549395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:37.549969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:37.552115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:37.568878 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:37.570944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ad22-a8ac-4631-8777-370e6a197424 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:37.584699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1305/2611] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:37.678432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ae4648ab484d41b7b5cd4aea22f5d3', 'request_id': 'req-c5e402be-e4c8-46a2-93ec-9e7f04f89024'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:37.698873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ae4648ab484d41b7b5cd4aea22f5d3', 'request_id': 'req-c5e402be-e4c8-46a2-93ec-9e7f04f89024'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:37.709745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ae4648ab484d41b7b5cd4aea22f5d3', 'request_id': 'req-c5e402be-e4c8-46a2-93ec-9e7f04f89024'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:37.853472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] The token cache is being invalidate because project 61ae4648ab484d41b7b5cd4aea22f5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:37.854778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:37.856332 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c5e402be-e4c8-46a2-93ec-9e7f04f89024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:37.859607 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1307/2612] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:37 2022] DELETE /identity/v3/projects/61ae4648ab484d41b7b5cd4aea22f5d3 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:37.867329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:37.871769 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:37.872509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:37.906179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:37.908523 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:37.982243 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:38.019793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1306/2613] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:38.029316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.030070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.030525 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.055021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:38.058786 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:38.073390 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1308/2614] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:38.078897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:38.095650 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14633014-60a2-447c-b566-3b0a0fd40daa None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:38.102577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:38.103908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.104528 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.106011 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.107671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:14:38.108522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:38.111931 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:38.135268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1307/2615] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:14:38.142092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:38.160480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401f952-065a-440f-9a36-b03f0ad8df7d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:38.170643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:38.171888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.172407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.172898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.174175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:38.178587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:38.303848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1309/2616] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:38 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:38.310922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:38.328774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddbc9d-d82e-422f-94fe-8479f9c6a606 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:38.425109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:38.426273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.426273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.426617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.427375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:38.430587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:38.561448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1308/2617] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:38 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:38.566142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:38.580719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2adaa-e454-4e47-a66e-f0930d3cb607 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:38.587990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:38.588885 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.589438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.590177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.591359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:38.595969 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:38.622961 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1310/2618] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:38.627025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:38.651042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65989616-4890-4f67-9540-596773bb7b8a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:38.653646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d3c16-f58a-429f-bd47-464bfadf4a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:38.654173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d3c16-f58a-429f-bd47-464bfadf4a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.655107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d3c16-f58a-429f-bd47-464bfadf4a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.655448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d3c16-f58a-429f-bd47-464bfadf4a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.656334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d3c16-f58a-429f-bd47-464bfadf4a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:38.659235 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d3c16-f58a-429f-bd47-464bfadf4a14 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:38.681947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1309/2619] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:14:38.691516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:38.701959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d00264-f0fe-4e61-b410-153da383b454 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:38.710099 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:38.711512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.712109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.712606 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] PATH_INFO: `/v3/projects/eecb0ffba1b440bc8cd1f4aa64b95b6e/users/555227be99ef4a93b6e1174515eae3db/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:38.719938 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:38.752859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecb0ffba1b440bc8cd1f4aa64b95b6e, user_id=555227be99ef4a93b6e1174515eae3db, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:38.758972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:38.968276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1311/2620] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:38 2022] PUT /identity/v3/projects/eecb0ffba1b440bc8cd1f4aa64b95b6e/users/555227be99ef4a93b6e1174515eae3db/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:38.978899 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:38.979211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:38.979489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:39.010627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] MFA Rules not processed for user `555227be99ef4a93b6e1174515eae3db`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:39.012451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:39.153558 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1310/2621] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:39.165126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:39.167266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:39.250220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174031b-c8e3-4153-ab9d-25e982b0f6ad admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:39.258020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4637080dea3a4349b53a3097f4610465', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:39.258020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:39.258020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:39.258020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:39.266534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:39.268883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:39.285060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1312/2622] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:43.175486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:43.175486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:43.305383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:43.332613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12a8b6-eab9-476d-a4fd-4c910b00a3c7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:43.336483 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:43.337220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:43.337725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:43.338109 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:43.341275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:43.366201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:43.368751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:43.478640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:43.478942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02db227-ed86-4ec3-8215-383eb6e40cdf service glance] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:43.480346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:43.485241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:43.486392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:43.486392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:43.486753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] PATH_INFO: `/v3/users/61401557e7624580b20de6e31c92f785` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:43.488070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61401557e7624580b20de6e31c92f785)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:43.490482 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:43.493197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1311/2623] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:14:43.788126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61401557e7624580b20de6e31c92f785'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:43.788903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61401557e7624580b20de6e31c92f785'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:43.806730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61401557e7624580b20de6e31c92f785'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:43.948312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61401557e7624580b20de6e31c92f785'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:43.965202 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-181347be-4b1b-4957-a1fa-653a3d058201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61401557e7624580b20de6e31c92f785'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:43.976749 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1313/2624] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:43 2022] DELETE /identity/v3/users/61401557e7624580b20de6e31c92f785 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:44.504885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:44.545706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cac9a-943c-494a-8433-345406a4cb99 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:44.667135 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:44.668940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:44.669305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:44.669590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] PATH_INFO: `/v3/projects/3ed58deb61194a5ba3f2d46eb7e5680c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:44.671083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed58deb61194a5ba3f2d46eb7e5680c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:44.673189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:44.898350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed58deb61194a5ba3f2d46eb7e5680c', 'request_id': 'req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:44.915929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed58deb61194a5ba3f2d46eb7e5680c', 'request_id': 'req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:45.064559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed58deb61194a5ba3f2d46eb7e5680c', 'request_id': 'req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:45.087673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] The token cache is being invalidate because project 3ed58deb61194a5ba3f2d46eb7e5680c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:45.087976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:45.089195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5e64b139-6cc8-44e4-b21a-f0bb7d6a672f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:45.091435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1312/2625] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:44 2022] DELETE /identity/v3/projects/3ed58deb61194a5ba3f2d46eb7e5680c => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:45.096799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:45.098172 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:45.098744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:45.118914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:45.136509 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:45.141517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:45.257958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1314/2626] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:45.267501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:45.267961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:45.268337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:45.302423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:45.304150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:45.321777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1313/2627] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:45.323340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:45.345108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59bb73-2429-4643-8cb8-7d63e1e06d13 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:45.355324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:45.356677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:45.357300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:45.357702 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:45.358600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:14:45.362910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:45.362910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:45.380127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1315/2628] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:14:45.387387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:45.417712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c874a-21eb-4d2b-983a-ac26efa8d2ad None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:45.423730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:45.426736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:45.426736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:45.426736 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:45.427834 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:45.431995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:45.472772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:45.684097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1314/2629] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:14:45 2022] POST /identity/v3/projects => generated 365 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:14:45.689158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:45.711433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b2e2-013c-434f-bbec-f33a1c821ff9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:45.847287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:45.848415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:45.848913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:45.849405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:45.850655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:45.856394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:45.978613 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1316/2630] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:14:45 2022] POST /identity/v3/users => generated 283 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:14:45.978965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:45.997121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13648581-0188-4bc3-bc2e-ced64574d24a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:46.007361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:46.008509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:46.009206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:46.010837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:46.011643 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:46.015175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:46.037289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1315/2631] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:46.043769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:46.065769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d613a2e-9739-42aa-af62-44999acb8891 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:46.074704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966074a-4de3-4c9e-a366-9d9669861c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:46.075419 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966074a-4de3-4c9e-a366-9d9669861c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:46.075834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966074a-4de3-4c9e-a366-9d9669861c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:46.076195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966074a-4de3-4c9e-a366-9d9669861c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:46.077080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966074a-4de3-4c9e-a366-9d9669861c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:46.079860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966074a-4de3-4c9e-a366-9d9669861c01 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:46.105227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1317/2632] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:14:46.110389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:46.129661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f5037-3c90-4138-be7a-42eaed43ef54 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:46.137059 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:46.138201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:46.138641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:46.139181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] PATH_INFO: `/v3/projects/42a1850055cf4d53993446a4cf402ae7/users/18c77f7c382a4c75a2854db7c5e45554/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:46.177143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a1850055cf4d53993446a4cf402ae7, user_id=18c77f7c382a4c75a2854db7c5e45554, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:46.180882 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:46.276127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1316/2633] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:14:46 2022] PUT /identity/v3/projects/42a1850055cf4d53993446a4cf402ae7/users/18c77f7c382a4c75a2854db7c5e45554/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:14:46.296450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:46.297056 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:46.297532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:46.340269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] MFA Rules not processed for user `18c77f7c382a4c75a2854db7c5e45554`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:46.341544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:46.428186 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1318/2634] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:14:46.441145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:46.442967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:46.535514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5f512-baff-4e05-b0ab-53ada6487299 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:46.542014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:46.543457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:46.543915 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:46.544381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:46.560713 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:46.562485 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:46.594130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1317/2635] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:55.689441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:55.692455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:55.698865 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:55.802798 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c02016-7bf7-4410-aef9-2b8fdfca5864 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:55.818335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:55.818335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:55.818335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:55.818335 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] PATH_INFO: `/v3/users/af1f81b83784472abc4b6ff4393afb7c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:55.819026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1f81b83784472abc4b6ff4393afb7c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:55.819026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:56.147752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1f81b83784472abc4b6ff4393afb7c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:56.148598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1f81b83784472abc4b6ff4393afb7c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:56.164534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1f81b83784472abc4b6ff4393afb7c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:56.256546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1f81b83784472abc4b6ff4393afb7c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:56.266604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1f81b83784472abc4b6ff4393afb7c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:56.273693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1319/2636] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:55 2022] DELETE /identity/v3/users/af1f81b83784472abc4b6ff4393afb7c => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:56.571863 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:56.589354 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:56.607180 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42527c2-58b1-4eb4-acf0-e0400ffac991 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:56.691282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:56.692067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:56.697225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:56.697225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] PATH_INFO: `/v3/projects/c1c061b824cb4a92a469061e7a67f967` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:56.697225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c061b824cb4a92a469061e7a67f967)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:56.697225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:56.919550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c061b824cb4a92a469061e7a67f967', 'request_id': 'req-bc3086a6-a0e8-407e-b893-c2acd6164563'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:56.929408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c061b824cb4a92a469061e7a67f967', 'request_id': 'req-bc3086a6-a0e8-407e-b893-c2acd6164563'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.029761 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c061b824cb4a92a469061e7a67f967', 'request_id': 'req-bc3086a6-a0e8-407e-b893-c2acd6164563'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.045839 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] The token cache is being invalidate because project c1c061b824cb4a92a469061e7a67f967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:57.046385 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.047406 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.049566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1318/2637] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:56 2022] DELETE /identity/v3/projects/c1c061b824cb4a92a469061e7a67f967 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:57.055470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:57.059093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:57.083719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4634d-6a50-4445-9f1b-8f0599f6e5d8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:57.188699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:57.189519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:57.190317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:57.190514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] PATH_INFO: `/v3/users/ac5265f19dc9475e889a5816c97ef624` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:57.193441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5265f19dc9475e889a5816c97ef624)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:57.202327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:57.253193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:57.497286 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5265f19dc9475e889a5816c97ef624'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.499156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5265f19dc9475e889a5816c97ef624'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.529192 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5265f19dc9475e889a5816c97ef624'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.644543 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5265f19dc9475e889a5816c97ef624'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.658745 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5265f19dc9475e889a5816c97ef624'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:57.671352 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1320/2638] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:57 2022] DELETE /identity/v3/users/ac5265f19dc9475e889a5816c97ef624 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:58.239247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:58.242190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:58.269452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3086a6-a0e8-407e-b893-c2acd6164563 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:58.361061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:58.361976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:58.362432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:58.362879 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] PATH_INFO: `/v3/projects/f394f5c330444b81b46cb0ba3f026cf0` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:58.364456 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f394f5c330444b81b46cb0ba3f026cf0)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:58.366940 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:58.638933 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:58.657461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f394f5c330444b81b46cb0ba3f026cf0', 'request_id': 'req-9107d22c-bdb0-480c-88f9-a66d6e027ad2'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:58.669873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f394f5c330444b81b46cb0ba3f026cf0', 'request_id': 'req-9107d22c-bdb0-480c-88f9-a66d6e027ad2'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:58.806710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f394f5c330444b81b46cb0ba3f026cf0', 'request_id': 'req-9107d22c-bdb0-480c-88f9-a66d6e027ad2'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:58.830338 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] The token cache is being invalidate because project f394f5c330444b81b46cb0ba3f026cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:14:58.831132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:58.832566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-9107d22c-bdb0-480c-88f9-a66d6e027ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:58.852758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1319/2639] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:14:58 2022] DELETE /identity/v3/projects/f394f5c330444b81b46cb0ba3f026cf0 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:59.152695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:59.154368 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:59.193555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eebcfc-be59-4f11-b289-ed3d18e045ba admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:59.294892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:59.296037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:59.296532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:59.297069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] PATH_INFO: `/v3/users/454e70d616544879b1c88796692dc8e3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:59.298751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454e70d616544879b1c88796692dc8e3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:59.300950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:59.592911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454e70d616544879b1c88796692dc8e3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:59.594184 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454e70d616544879b1c88796692dc8e3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:59.623391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454e70d616544879b1c88796692dc8e3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:59.716807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454e70d616544879b1c88796692dc8e3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:59.727820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454e70d616544879b1c88796692dc8e3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:14:59.736178 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1321/2640] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:14:59 2022] DELETE /identity/v3/users/454e70d616544879b1c88796692dc8e3 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:14:59.741983 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:59.742844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:59.743425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:59.768222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:14:59.769479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:59.851869 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1320/2641] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:14:59.865680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:59.870203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:59.940979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:00.004911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8ded5-c72b-4d43-ac8b-809f21a06929 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:00.015103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca65e7-2091-4fc3-8b68-b69492611400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:00.022667 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca65e7-2091-4fc3-8b68-b69492611400 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:00.023272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca65e7-2091-4fc3-8b68-b69492611400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:00.026347 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca65e7-2091-4fc3-8b68-b69492611400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:00.047873 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca65e7-2091-4fc3-8b68-b69492611400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:00.050718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca65e7-2091-4fc3-8b68-b69492611400 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:00.072109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1322/2642] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:00.730362 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:00.738903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:00.754690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb244e3-a892-4caf-aca7-c388631e94dd None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:00.759674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:00.762600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:00.763368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:00.764450 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] PATH_INFO: `/v3/projects/e0a37bc3a9a64bd6bb87348f829b2566` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:00.766435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a37bc3a9a64bd6bb87348f829b2566)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:00.773475 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:00.960091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a37bc3a9a64bd6bb87348f829b2566', 'request_id': 'req-a4262950-c71d-4589-b812-0bddaf1bd80d'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:00.968566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a37bc3a9a64bd6bb87348f829b2566', 'request_id': 'req-a4262950-c71d-4589-b812-0bddaf1bd80d'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:01.063812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a37bc3a9a64bd6bb87348f829b2566', 'request_id': 'req-a4262950-c71d-4589-b812-0bddaf1bd80d'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:01.104124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] The token cache is being invalidate because project e0a37bc3a9a64bd6bb87348f829b2566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:01.104512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:01.105797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-a4262950-c71d-4589-b812-0bddaf1bd80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:01.107445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1321/2643] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:00 2022] DELETE /identity/v3/projects/e0a37bc3a9a64bd6bb87348f829b2566 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:01.114644 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.115197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.115384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.142850 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:01.144002 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:01.223268 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1323/2644] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:01.232037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.232314 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.232636 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.256627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:01.256627 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:01.266053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1322/2645] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:01.273429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:01.295205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e5505-b0a2-424a-a196-70737fb0cd9a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:01.304826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:01.306022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.306527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.307096 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.308898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:01.309321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:01.311883 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:01.338311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1324/2646] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:15:01.343014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:01.354099 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee8085-8c4c-4603-87ab-4dbeb764f6c3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:01.359454 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:01.360312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.360777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.361222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.361889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:01.363911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:01.464323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1323/2647] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:01 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:01.469049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:01.480008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb44a6-ef17-426e-a8be-a3a1a825c5d5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:01.580741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:01.581821 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.582377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.582884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.584100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:01.588011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:01.724892 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1325/2648] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:01 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:01.730707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:01.742024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f7a5a-2809-4271-8710-143530b1d8c2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:01.748031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:01.748705 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.748974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.749254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.749964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:01.752215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:01.768680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1324/2649] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:01.773197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:01.789550 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b28d-2a27-4e20-8d2b-b5deb752e1e6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:01.796311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d6cfb-5070-4b69-88f1-91bf479efc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:01.797021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6cfb-5070-4b69-88f1-91bf479efc03 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.797378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6cfb-5070-4b69-88f1-91bf479efc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.797668 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6cfb-5070-4b69-88f1-91bf479efc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.798363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d6cfb-5070-4b69-88f1-91bf479efc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:01.800211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d6cfb-5070-4b69-88f1-91bf479efc03 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:01.823376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1326/2650] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:01.828568 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:01.837100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:01.843642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45536946-6aa3-49da-8faa-fcc6efffbd7e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:01.849364 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:01.850191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:01.850588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:01.850936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] PATH_INFO: `/v3/projects/923c682cd63f4c839f556d4bf26e8d27/users/b7a43a70dc8e4ba88c079f1f21a20cde/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:01.870381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] RBAC: Authorizing `identity:create_grant(project_id=923c682cd63f4c839f556d4bf26e8d27, user_id=b7a43a70dc8e4ba88c079f1f21a20cde, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:01.882551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.008976 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1325/2651] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:01 2022] PUT /identity/v3/projects/923c682cd63f4c839f556d4bf26e8d27/users/b7a43a70dc8e4ba88c079f1f21a20cde/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:02.024257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.024500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.024715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.060278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] MFA Rules not processed for user `b7a43a70dc8e4ba88c079f1f21a20cde`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:02.061765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:02.085923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:02.158542 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1327/2652] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:02.164818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:02.293492 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b624f0-cd53-4db6-b12d-10f59e10e21a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:02.307993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:02.309118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.309662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.310686 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.311522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:02.315061 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.442715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1326/2653] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:02 2022] POST /identity/v3/projects => generated 383 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:02.444166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:02.467926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68245279-c2c2-45ea-978d-0e0d897637c0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:02.537517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:02.538158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.538562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.539067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.539932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:02.542831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.636067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:02.637989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:02.653445 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb863f-e491-429e-95de-a4e2464bde31 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:02.659690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:02.663710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.663710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.663710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] PATH_INFO: `/v3/users/18c77f7c382a4c75a2854db7c5e45554` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.663710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c77f7c382a4c75a2854db7c5e45554)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:02.664645 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.695589 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1328/2654] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:02 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:02.710700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:02.734103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2700e4-830b-4203-9191-d84e3b740b32 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:02.742954 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:02.743875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.744299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.744912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.746554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:02.748676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.765211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1329/2655] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:02.770422 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:02.790967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fea4d0-8cd4-4d14-9b45-41605eec5dff admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:02.804217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:02.804997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.808384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.808384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.808384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:02.813049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.823681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1330/2656] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:02.829465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:02.848945 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820f98a-0cea-493c-9f9c-0c9d32acd623 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:02.857849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5921d78-f25c-4812-802a-6c7dee63ed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:02.859502 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921d78-f25c-4812-802a-6c7dee63ed56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:02.860139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921d78-f25c-4812-802a-6c7dee63ed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:02.861255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921d78-f25c-4812-802a-6c7dee63ed56 admin admin] PATH_INFO: `/v3/projects/b49b14278b2a484d967d522aa20ae1cf/users/e12a822f333e4003ae001b193a36554a/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:02.892092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5921d78-f25c-4812-802a-6c7dee63ed56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49b14278b2a484d967d522aa20ae1cf, user_id=e12a822f333e4003ae001b193a36554a, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:02.897662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5921d78-f25c-4812-802a-6c7dee63ed56 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:02.984070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c77f7c382a4c75a2854db7c5e45554'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:03.049367 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1331/2657] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:02 2022] PUT /identity/v3/projects/b49b14278b2a484d967d522aa20ae1cf/users/e12a822f333e4003ae001b193a36554a/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:03.060788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.061085 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.061351 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.071555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c77f7c382a4c75a2854db7c5e45554'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:03.103943 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c77f7c382a4c75a2854db7c5e45554'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:03.104901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] MFA Rules not processed for user `e12a822f333e4003ae001b193a36554a`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:03.105658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c77f7c382a4c75a2854db7c5e45554'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:03.106339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:03.122071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-60df7c66-2861-4f5b-b9f0-41a843dcc165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c77f7c382a4c75a2854db7c5e45554'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:03.133567 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1327/2658] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:02 2022] DELETE /identity/v3/users/18c77f7c382a4c75a2854db7c5e45554 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:03.140760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.141304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.141719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.166007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:03.167610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:03.186742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:03.220246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1332/2659] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:03.220548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:03.225959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:03.227769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:03.269083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1328/2660] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:03.284786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:03.286559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:03.327055 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79575684-2646-4996-904f-4d8cc2f49cb4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:03.334027 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:03.335030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.335487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.336097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.337205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:03.342297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:03.375607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd54b0-24a9-4887-a4be-d9b982cdf255 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:03.379734 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:03.380451 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.380756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.381012 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.392772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:03.394306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:03.407092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1329/2661] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:03.502388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1333/2662] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:03 2022] POST /identity/v3/projects => generated 383 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:03.507604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:03.520951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6b8cc-d863-44d5-9a8c-d94393e07ca0 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:03.601121 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:03.601818 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.602133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.602411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.603069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:03.605698 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:03.820687 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1330/2663] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:03 2022] POST /identity/v3/users => generated 298 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:03.823870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:03.836323 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e64af-bea8-4648-b982-87976aabd79d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:03.842452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:03.843125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.843440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.843748 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.844512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:03.846442 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:03.872359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1334/2664] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:03.882553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:03.897856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a4fd3-a65e-4fe7-9301-12baa5cd687b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:03.907084 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:03.908494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:03.910199 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:03.910747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] PATH_INFO: `/v3/projects/265cc0fe67bd477ca53de249cd3f7917/users/146d7433e9614b26b4c7a03d2ba19716/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:03.940580 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=265cc0fe67bd477ca53de249cd3f7917, user_id=146d7433e9614b26b4c7a03d2ba19716, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:03.948889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.066256 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1331/2665] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:03 2022] PUT /identity/v3/projects/265cc0fe67bd477ca53de249cd3f7917/users/146d7433e9614b26b4c7a03d2ba19716/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:04.070513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:04.084222 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913f125-ca63-46ec-a8e3-9c37861530c2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:04.150271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:04.155131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:04.156857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:04.157803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.158507 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.158772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.159689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:04.161455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.178966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1335/2666] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:04.186979 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4cdbe-148e-447e-a540-9cac380ab3aa admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:04.191611 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:04.193832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:04.195207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.195572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.195927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] PATH_INFO: `/v3/projects/42a1850055cf4d53993446a4cf402ae7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.199044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a1850055cf4d53993446a4cf402ae7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:04.202079 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.216853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abad7-c065-4ad4-9959-8f2f90ebd5ea admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:04.225432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c11ee-b421-4d3c-99f4-e302b9955c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:04.226770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c11ee-b421-4d3c-99f4-e302b9955c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.227260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c11ee-b421-4d3c-99f4-e302b9955c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.227756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c11ee-b421-4d3c-99f4-e302b9955c98 admin admin] PATH_INFO: `/v3/projects/265cc0fe67bd477ca53de249cd3f7917/users/146d7433e9614b26b4c7a03d2ba19716/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.231843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c11ee-b421-4d3c-99f4-e302b9955c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=265cc0fe67bd477ca53de249cd3f7917, user_id=146d7433e9614b26b4c7a03d2ba19716, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:04.237534 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c11ee-b421-4d3c-99f4-e302b9955c98 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.255967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1336/2667] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:04 2022] PUT /identity/v3/projects/265cc0fe67bd477ca53de249cd3f7917/users/146d7433e9614b26b4c7a03d2ba19716/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:04.270839 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.271801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.272308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.325927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] MFA Rules not processed for user `146d7433e9614b26b4c7a03d2ba19716`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:04.328547 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:04.381247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:04.391555 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a1850055cf4d53993446a4cf402ae7', 'request_id': 'req-6219906b-8ad8-4e1c-825f-716fedfa2a3c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:04.403797 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:04.404753 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a1850055cf4d53993446a4cf402ae7', 'request_id': 'req-6219906b-8ad8-4e1c-825f-716fedfa2a3c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:04.446287 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1337/2668] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:04.468145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:04.469610 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:04.504792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a1850055cf4d53993446a4cf402ae7', 'request_id': 'req-6219906b-8ad8-4e1c-825f-716fedfa2a3c'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:04.525515 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] The token cache is being invalidate because project 42a1850055cf4d53993446a4cf402ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:04.526044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:04.527145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6219906b-8ad8-4e1c-825f-716fedfa2a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:04.529156 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1332/2669] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:04 2022] DELETE /identity/v3/projects/42a1850055cf4d53993446a4cf402ae7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:04.535214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1994f0-8daf-4700-bca3-441055928cf4 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.535543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1994f0-8daf-4700-bca3-441055928cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.535874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1994f0-8daf-4700-bca3-441055928cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.552046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7a19c-fd3b-4ff1-8dfe-97d865aefe33 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:04.565000 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7c1994f0-8daf-4700-bca3-441055928cf4 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:04.567048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7c1994f0-8daf-4700-bca3-441055928cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:04.658536 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:04.659206 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.659861 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.660372 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.663239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:04.694292 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1333/2670] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:04.704777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.705935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.706468 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.737356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:04.738997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:04.749399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1334/2671] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:04.754721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:04.757467 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:04.759024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.767772 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f09fb2-6461-4fda-a35d-7fc5277a10bf None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:04.773478 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:04.774298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1338/2672] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:04.776701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.777005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.777342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.778229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:04.778569 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:04.780351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.796684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1335/2673] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:15:04.802232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:04.821573 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac348-03fd-4225-a881-719e50b372eb service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:04.830246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:04.831755 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:04.832246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:04.832727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:04.833879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:04.837732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:04.979043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1339/2674] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:04 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:04.987359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:05.006921 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f6efe-6935-49a0-b856-10d416349eb8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:05.081693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:05.082852 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:05.083204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:05.083544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:05.084399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:05.086610 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:05.212388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1336/2675] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:04 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:05.218706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:05.231431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f660d-ccab-4551-9a8d-8e0de3e4e0c7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:05.239359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:05.240373 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:05.243663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:05.243663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:05.243663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:05.246026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:05.266563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1340/2676] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:05.271228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:05.283343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183023d-327a-4b43-ac7e-54f7f89aeb97 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:05.289016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e0d6d-905e-43ca-b9a1-162a65b43463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:05.289706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0d6d-905e-43ca-b9a1-162a65b43463 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:05.290110 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0d6d-905e-43ca-b9a1-162a65b43463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:05.290463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0d6d-905e-43ca-b9a1-162a65b43463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:05.291210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e0d6d-905e-43ca-b9a1-162a65b43463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:05.293930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e0d6d-905e-43ca-b9a1-162a65b43463 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:05.309105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1337/2677] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:05.314276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:05.326154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6232a-20f9-434f-bb5d-ae1d8e098751 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:05.331824 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:05.332580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:05.332912 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:05.333242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] PATH_INFO: `/v3/projects/a7dd58dd75ac4de986e5790ee1fb9dcc/users/299992c07bcf40af9d44d0a26aa9a429/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:05.345188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:05.355698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7dd58dd75ac4de986e5790ee1fb9dcc, user_id=299992c07bcf40af9d44d0a26aa9a429, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:05.358563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:05.502732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1341/2678] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:05 2022] PUT /identity/v3/projects/a7dd58dd75ac4de986e5790ee1fb9dcc/users/299992c07bcf40af9d44d0a26aa9a429/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:05.515072 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:05.515532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:05.515969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:05.563205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] MFA Rules not processed for user `299992c07bcf40af9d44d0a26aa9a429`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:05.564508 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:05.597460 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:05.701824 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1338/2679] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:05.721197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:05.723689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:05.892174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636b8f6-e524-4d49-b00d-c818bff9890a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:05.899897 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e85c8-3396-4243-a6da-eae1fd210589 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:05.900664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e85c8-3396-4243-a6da-eae1fd210589 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:05.901628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e85c8-3396-4243-a6da-eae1fd210589 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:05.902261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e85c8-3396-4243-a6da-eae1fd210589 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:05.915697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e85c8-3396-4243-a6da-eae1fd210589 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:05.918529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e85c8-3396-4243-a6da-eae1fd210589 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:05.931600 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1342/2680] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:06.243263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:06.320146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6855-7e8c-40d4-90d8-aa4d32f97895 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:06.326023 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:06.326755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:06.327228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:06.327618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] PATH_INFO: `/v3/users/299992c07bcf40af9d44d0a26aa9a429` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:06.329019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] RBAC: Authorizing `identity:delete_user(user_id=299992c07bcf40af9d44d0a26aa9a429)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:06.331601 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:06.581656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299992c07bcf40af9d44d0a26aa9a429'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:06.714417 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299992c07bcf40af9d44d0a26aa9a429'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:06.732510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299992c07bcf40af9d44d0a26aa9a429'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:06.733270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299992c07bcf40af9d44d0a26aa9a429'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:06.741678 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299992c07bcf40af9d44d0a26aa9a429'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:06.749179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1339/2681] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:06 2022] DELETE /identity/v3/users/299992c07bcf40af9d44d0a26aa9a429 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:06.754275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:06.755234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:06.755234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:06.781781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:06.784049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:06.888901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1343/2682] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:06.899781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:06.901998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:06.956116 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:06.974472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aed81d-64f6-4f9f-beb1-63363180081a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:06.979323 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4ff15-7e25-4c29-a978-a30963231651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:06.980038 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4ff15-7e25-4c29-a978-a30963231651 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:06.980366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4ff15-7e25-4c29-a978-a30963231651 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:06.980701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4ff15-7e25-4c29-a978-a30963231651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:06.991306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4ff15-7e25-4c29-a978-a30963231651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:06.992848 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4ff15-7e25-4c29-a978-a30963231651 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:07.006524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1340/2683] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:07.709943 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:07.712246 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:07.740171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde19a6e-c40a-4b41-b8ae-3416508dc449 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:07.766183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:07.766183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:07.766183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:07.766183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] PATH_INFO: `/v3/projects/a7dd58dd75ac4de986e5790ee1fb9dcc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:07.767264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7dd58dd75ac4de986e5790ee1fb9dcc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:07.770841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:07.978894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:07.981311 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7dd58dd75ac4de986e5790ee1fb9dcc', 'request_id': 'req-70009353-8e45-409a-9252-1d3fde083a04'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:07.988843 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7dd58dd75ac4de986e5790ee1fb9dcc', 'request_id': 'req-70009353-8e45-409a-9252-1d3fde083a04'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:07.998441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7dd58dd75ac4de986e5790ee1fb9dcc', 'request_id': 'req-70009353-8e45-409a-9252-1d3fde083a04'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:08.109362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] The token cache is being invalidate because project a7dd58dd75ac4de986e5790ee1fb9dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:08.110121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:08.111312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-70009353-8e45-409a-9252-1d3fde083a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:08.114713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1344/2684] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:07 2022] DELETE /identity/v3/projects/a7dd58dd75ac4de986e5790ee1fb9dcc => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:08.123425 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.124299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.124504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.153298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:08.154598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:08.227436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1341/2685] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:08.239008 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.239300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.239554 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.264044 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:08.265336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:08.274453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1345/2686] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:08.280185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:08.290389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c654ce-43a6-4983-ad43-dcc0f25828ce None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:08.295288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:08.295922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.296206 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.296471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.297143 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:08.297479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:08.299160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:08.314608 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1342/2687] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:15:08.320890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:08.332265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5e5ce-c3c2-4fbc-ad91-68e4f48a83cd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:08.337262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:08.337890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.338204 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.339374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.340097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:08.342194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:08.458579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1346/2688] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:08 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:08.464105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:08.475342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc902c-87f3-4680-8da1-195940c1aafb admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:08.548361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:08.549044 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.549416 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.549752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.550584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:08.552949 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:08.636547 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1343/2689] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:08 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:08.641293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:08.653454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240e611-f56f-44c3-b9e9-74c9dad50daa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:08.658715 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:08.659400 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.659675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.659931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.660585 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:08.662278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:08.675254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1347/2690] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:08.682532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:08.694126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697c5b2-6ee6-48a3-ba4b-e1b5e833a00a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:08.700259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ea856-d705-4464-ab82-161b35506a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:08.700903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ea856-d705-4464-ab82-161b35506a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.701190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ea856-d705-4464-ab82-161b35506a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.701467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ea856-d705-4464-ab82-161b35506a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.702171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ea856-d705-4464-ab82-161b35506a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:08.703833 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ea856-d705-4464-ab82-161b35506a23 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:08.717213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1344/2691] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:08.726738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:08.739195 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a5076-2b83-4b00-9d86-7c8aed840ed9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:08.744348 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:08.745018 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.745299 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.745616 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] PATH_INFO: `/v3/projects/966da54c52d64018a7a84f0f45bebb73/users/0854939703534ebb9cf20474167ce68b/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.763399 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966da54c52d64018a7a84f0f45bebb73, user_id=0854939703534ebb9cf20474167ce68b, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:08.766146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:08.858112 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1348/2692] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:08 2022] PUT /identity/v3/projects/966da54c52d64018a7a84f0f45bebb73/users/0854939703534ebb9cf20474167ce68b/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:08.878097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:08.878646 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:08.879163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:08.929300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] MFA Rules not processed for user `0854939703534ebb9cf20474167ce68b`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:08.929300 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:08.979588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:09.027893 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1345/2693] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:09.035232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.114011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868f07d-9172-4de5-b457-133595261f25 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.119130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.119851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.120074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.120428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.121120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.123386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.229319 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1349/2694] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:09 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:09.234223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.259656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5a08e-fa78-44f5-b509-53093dc01a46 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.351617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.354681 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:09.409638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.410597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.411065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.411532 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.412618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.416287 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.455982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d12e5-8ab5-470b-b066-b8f2c59ba0dd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.460914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.461670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.462066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.462416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.465472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:09.489115 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.491914 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.502602 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:09.509788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1350/2695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:09.512139 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.515003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:09.544297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1346/2696] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:09 2022] POST /identity/v3/users => generated 297 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:09.550114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.569319 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ab420-4407-41f9-8245-9a4234f2ed1a admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.589500 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.591566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.591566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.592152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.593516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.597991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.623929 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1347/2697] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:09.625648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.643927 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63cc5c-8777-4bc8-939c-ec6fab084dfb admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.653107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.654248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7334a-194c-413a-a3fe-7b0602e7f713 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.654423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.654781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.655266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] PATH_INFO: `/v3/projects/2b96f0958fd14ee2ac9b1318ead77017/users/5444dbf072e34b29aad12ee26cbd5efa/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.664753 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.668597 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.671126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.671126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.673933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:09.689377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b96f0958fd14ee2ac9b1318ead77017, user_id=5444dbf072e34b29aad12ee26cbd5efa, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.695043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.780281 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.782505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.799081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1351/2698] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:09.838658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1348/2699] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:09 2022] PUT /identity/v3/projects/2b96f0958fd14ee2ac9b1318ead77017/users/5444dbf072e34b29aad12ee26cbd5efa/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:09.844508 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:09.860723 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a38c3-158a-434e-a62e-97edef1b96fd service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:09.956527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48450a94-ed47-4c1c-a843-06e9191a0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:09.957572 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48450a94-ed47-4c1c-a843-06e9191a0fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:09.958336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48450a94-ed47-4c1c-a843-06e9191a0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:09.959782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48450a94-ed47-4c1c-a843-06e9191a0fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:09.961173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48450a94-ed47-4c1c-a843-06e9191a0fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:09.965350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48450a94-ed47-4c1c-a843-06e9191a0fb8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:09.987070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1352/2700] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:09.992509 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:10.008597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32a84-1fdd-4945-b275-6fc93a9f99b2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:10.018895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:10.020342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:10.020696 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:10.021114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] PATH_INFO: `/v3/projects/2b96f0958fd14ee2ac9b1318ead77017/users/5444dbf072e34b29aad12ee26cbd5efa/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:10.024910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b96f0958fd14ee2ac9b1318ead77017, user_id=5444dbf072e34b29aad12ee26cbd5efa, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:10.032441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:10.049767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1349/2701] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:09 2022] PUT /identity/v3/projects/2b96f0958fd14ee2ac9b1318ead77017/users/5444dbf072e34b29aad12ee26cbd5efa/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:10.069631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:10.070188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:10.070705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:10.132492 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] MFA Rules not processed for user `5444dbf072e34b29aad12ee26cbd5efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:10.134957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:10.252741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1353/2702] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:10.269527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:10.271971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:10.380742 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08277c-90aa-4f11-ad92-4b646999422d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:10.386304 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:10.387003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:10.387257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:10.387537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:10.400040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:10.401703 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:10.413787 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1350/2703] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:10.471125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:10.903638 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:10.918260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd39d8b-be88-4071-9a45-f4f0aa67b881 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:10.924308 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:10.925997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:10.925997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:10.926361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:11.060847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:11.064003 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:11.086706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1354/2704] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:12.204225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:12.221371 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1d3d4-17aa-428f-892e-374cedc14c2e service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:12.228510 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:12.229674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:12.230215 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:12.230791 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:12.234078 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:12.281505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:12.403985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:12.407115 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:12.429732 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1351/2705] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:15.153871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:15.156425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:15.252765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5a73-ae06-4c23-af61-8266378a574b service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:15.257678 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:15.258382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:15.258669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:15.258952 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] PATH_INFO: `/v3/users/7ae66df61a764a42a1b1b15f54d5c34b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:15.260410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae66df61a764a42a1b1b15f54d5c34b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:15.262461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:15.467493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae66df61a764a42a1b1b15f54d5c34b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:15.468278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae66df61a764a42a1b1b15f54d5c34b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:15.490125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae66df61a764a42a1b1b15f54d5c34b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:15.890700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae66df61a764a42a1b1b15f54d5c34b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:15.897209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:15.901624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae66df61a764a42a1b1b15f54d5c34b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:15.910940 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1355/2706] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:15 2022] DELETE /identity/v3/users/7ae66df61a764a42a1b1b15f54d5c34b => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:16.241529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:16.243383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:16.261651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:16.264438 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:16.294368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8200c26-b1e4-42e6-a487-eed1ed141e7e service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:16.419769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bbdd9-982e-4679-9eb1-6dcf3a331186 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:16.421141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:16.422321 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:16.422908 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:16.423501 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] PATH_INFO: `/v3/users/555227be99ef4a93b6e1174515eae3db` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:16.425724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555227be99ef4a93b6e1174515eae3db)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:16.426101 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:16.426853 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:16.427221 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:16.427580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] PATH_INFO: `/v3/projects/90a84500a899479aabfa3aeb72b4c4ae` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:16.429020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a84500a899479aabfa3aeb72b4c4ae)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:16.429704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:16.431322 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:16.678521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a84500a899479aabfa3aeb72b4c4ae', 'request_id': 'req-80d9f7e2-400e-417f-96a4-5fc067f8aa37'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.694356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a84500a899479aabfa3aeb72b4c4ae', 'request_id': 'req-80d9f7e2-400e-417f-96a4-5fc067f8aa37'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.724565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a84500a899479aabfa3aeb72b4c4ae', 'request_id': 'req-80d9f7e2-400e-417f-96a4-5fc067f8aa37'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.851459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555227be99ef4a93b6e1174515eae3db'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.890578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] The token cache is being invalidate because project 90a84500a899479aabfa3aeb72b4c4ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:16.890907 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.891646 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-80d9f7e2-400e-417f-96a4-5fc067f8aa37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.893265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1356/2707] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:16 2022] DELETE /identity/v3/projects/90a84500a899479aabfa3aeb72b4c4ae => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:16.898782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba2784-5ce6-4c2e-be78-1137975529f5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:16.899127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba2784-5ce6-4c2e-be78-1137975529f5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:16.899356 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba2784-5ce6-4c2e-be78-1137975529f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:16.924213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e0ba2784-5ce6-4c2e-be78-1137975529f5 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:16.925485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e0ba2784-5ce6-4c2e-be78-1137975529f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:16.976207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555227be99ef4a93b6e1174515eae3db'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.993939 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555227be99ef4a93b6e1174515eae3db'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:16.995750 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555227be99ef4a93b6e1174515eae3db'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:17.004765 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-28e2e1bf-efde-440d-8ee2-b6b18481e5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555227be99ef4a93b6e1174515eae3db'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:17.012411 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1352/2708] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:16 2022] DELETE /identity/v3/users/555227be99ef4a93b6e1174515eae3db => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:17.013682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1357/2709] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:17.018810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.019146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.019570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.025465 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.026742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.027257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.055588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:17.058992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:17.059240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:17.060540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:17.112055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:17.157763 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:17.160995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1358/2710] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:17.165816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.190782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1353/2711] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:17.191065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57cc93-75ce-460f-b7a5-2d5de1adfd95 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.191065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.191065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.191065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.192655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.192655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:17.192655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.192655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:17.199770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.202020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:17.207766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1359/2712] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:15:17.214331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.228444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4050-eeaa-4879-b9ce-048a789fa6a2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.235493 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.236277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.236657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.237034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.237790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.240292 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:17.287474 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb228f1-b64d-4c25-8ae6-4d89dd4648ff None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.298311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.298964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.299248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.299521 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.311634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.313104 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:17.328764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1354/2713] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:17.360244 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1360/2714] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:17 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:17.365653 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.380227 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575b08-5e76-4c9a-b593-335fda36a8ed service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.468679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.469313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.469597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.469875 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.470560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.472618 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:17.709437 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1355/2715] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:17 2022] POST /identity/v3/users => generated 303 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:17.714320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.727151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23bfd5-6691-4a7e-8440-f037e31821a6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.733609 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.734744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.735587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.736023 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.736956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.739291 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:17.756295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1361/2716] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:17.762982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.781699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967681-99dd-4b2a-945d-c0b8f96c84ca admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.790313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.791043 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.791798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.792081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.793183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.795171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:17.813095 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1356/2717] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:17.818005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:17.831387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1d6d6-2f9f-450d-ae36-a4a2ba610c4f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:17.837320 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c434f-6305-44b4-83f8-edaa85d73003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:17.838413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c434f-6305-44b4-83f8-edaa85d73003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:17.839425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c434f-6305-44b4-83f8-edaa85d73003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:17.839425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c434f-6305-44b4-83f8-edaa85d73003 admin admin] PATH_INFO: `/v3/projects/c18ce7a0f3f7423a847de666cfd4d5b6/users/95c4751b333843ad80c7abcffe82a09f/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:17.860789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c434f-6305-44b4-83f8-edaa85d73003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18ce7a0f3f7423a847de666cfd4d5b6, user_id=95c4751b333843ad80c7abcffe82a09f, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:17.863801 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c434f-6305-44b4-83f8-edaa85d73003 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:18.006953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1362/2718] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:17 2022] PUT /identity/v3/projects/c18ce7a0f3f7423a847de666cfd4d5b6/users/95c4751b333843ad80c7abcffe82a09f/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:18.019154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:18.021623 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:18.026073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:18.026073 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:18.026955 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:18.045002 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d66df-02f4-47fd-b5be-d312ea339ce8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:18.080229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] MFA Rules not processed for user `95c4751b333843ad80c7abcffe82a09f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:18.082474 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:18.131124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:18.143624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:18.144699 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:18.145127 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:18.145776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] PATH_INFO: `/v3/projects/eecb0ffba1b440bc8cd1f4aa64b95b6e` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:18.148111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eecb0ffba1b440bc8cd1f4aa64b95b6e)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:18.151408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:18.180211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1363/2719] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:18 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:18.187223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:18.202662 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5e0e6-d059-4867-9f1c-b0c0a85cf463 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:18.209421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:18.212540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:18.212540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:18.212540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:18.212540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:18.214544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:18.306881 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eecb0ffba1b440bc8cd1f4aa64b95b6e', 'request_id': 'req-da35c33d-3da6-47da-8bf2-71e8d108dd9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:18.317672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eecb0ffba1b440bc8cd1f4aa64b95b6e', 'request_id': 'req-da35c33d-3da6-47da-8bf2-71e8d108dd9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:18.334854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1364/2720] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:18 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:18.340214 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:18.368810 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:18.413331 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fee3b-d07a-4d52-a198-9c12bf01030c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:18.415651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eecb0ffba1b440bc8cd1f4aa64b95b6e', 'request_id': 'req-da35c33d-3da6-47da-8bf2-71e8d108dd9e'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:18.420162 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:18.420916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:18.421275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:18.421689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:18.422622 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:18.425155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:18.429780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] The token cache is being invalidate because project eecb0ffba1b440bc8cd1f4aa64b95b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:18.430134 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:18.430959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:18.433181 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1357/2721] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:18 2022] DELETE /identity/v3/projects/eecb0ffba1b440bc8cd1f4aa64b95b6e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:18.568459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1365/2722] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:18 2022] POST /identity/v3/users => generated 308 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:18.571279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:18.574266 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:18.700056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35c33d-3da6-47da-8bf2-71e8d108dd9e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:18.710906 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:18.712048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:18.712586 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:18.713613 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:18.714842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:18.718090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:18.739091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1358/2723] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:18.744685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:18.763730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dbc58-8c8a-48d1-8673-6267f3bc27f8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:18.771886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:18.773066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:18.773588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:18.774154 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] PATH_INFO: `/v3/projects/2f9d3976ff46464bb63c961f942d5184/users/4805c285aa344d9ca77663074045e622/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:18.802946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9d3976ff46464bb63c961f942d5184, user_id=4805c285aa344d9ca77663074045e622, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:18.808819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:18.983436 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1366/2724] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:18 2022] PUT /identity/v3/projects/2f9d3976ff46464bb63c961f942d5184/users/4805c285aa344d9ca77663074045e622/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:18.988334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:19.004948 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20c429-861a-4f2f-bba2-578d3c2d04f8 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:19.083973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e11e42-4f88-46bf-9321-2217dc415348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:19.085016 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e11e42-4f88-46bf-9321-2217dc415348 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:19.085521 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e11e42-4f88-46bf-9321-2217dc415348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:19.086040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e11e42-4f88-46bf-9321-2217dc415348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:19.087222 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e11e42-4f88-46bf-9321-2217dc415348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:19.091122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e11e42-4f88-46bf-9321-2217dc415348 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:19.111171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1359/2725] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:19.115460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:19.131491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383c269-f9e6-4051-80e5-9b8f436edf15 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:19.138984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:19.139729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:19.140070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:19.140461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] PATH_INFO: `/v3/projects/2f9d3976ff46464bb63c961f942d5184/users/4805c285aa344d9ca77663074045e622/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:19.142878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9d3976ff46464bb63c961f942d5184, user_id=4805c285aa344d9ca77663074045e622, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:19.145848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:19.160041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1367/2726] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:19 2022] PUT /identity/v3/projects/2f9d3976ff46464bb63c961f942d5184/users/4805c285aa344d9ca77663074045e622/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:19.171057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:19.171349 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:19.171640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:19.206424 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] MFA Rules not processed for user `4805c285aa344d9ca77663074045e622`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:19.207695 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:19.213845 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:19.290027 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1360/2727] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:19 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:19.303377 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:19.305339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:19.380510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd471-1457-47a2-affa-0c72530c6971 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:19.385653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb879d0b-1e06-4d85-8a7d-cf44a8ad9399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:19.386345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb879d0b-1e06-4d85-8a7d-cf44a8ad9399 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:19.386628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb879d0b-1e06-4d85-8a7d-cf44a8ad9399 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:19.386898 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb879d0b-1e06-4d85-8a7d-cf44a8ad9399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:19.397683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb879d0b-1e06-4d85-8a7d-cf44a8ad9399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:19.399177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb879d0b-1e06-4d85-8a7d-cf44a8ad9399 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:19.410779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1368/2728] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:19 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:19.569505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:19.658641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d6140-b254-4c4c-a93d-4d05ab718a23 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:19.664220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:19.667329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:19.667329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:19.667329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] PATH_INFO: `/v3/users/95c4751b333843ad80c7abcffe82a09f` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:19.667329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c4751b333843ad80c7abcffe82a09f)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:19.669820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:20.038757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c4751b333843ad80c7abcffe82a09f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.147164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c4751b333843ad80c7abcffe82a09f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.163507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c4751b333843ad80c7abcffe82a09f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.164264 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c4751b333843ad80c7abcffe82a09f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.172690 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c4751b333843ad80c7abcffe82a09f'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.179662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1361/2729] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:19 2022] DELETE /identity/v3/users/95c4751b333843ad80c7abcffe82a09f => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:20.185231 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:20.185707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:20.186146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:20.209931 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:20.211234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:20.284294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1369/2730] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:20.297275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:20.299230 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:20.441197 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:20.442981 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:20.459895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c556ee-083b-45f6-b846-8e61805e51bd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:20.466283 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:20.466971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:20.467300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:20.467910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] PATH_INFO: `/v3/users/b7a43a70dc8e4ba88c079f1f21a20cde` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:20.469010 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:20.469738 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a43a70dc8e4ba88c079f1f21a20cde)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:20.471960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:20.490187 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:20.511387 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84a73-7b6a-4795-aea9-d004f0cb286d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:20.515438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823853c6-d338-4c76-895d-81f76dc3ec77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:20.516106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823853c6-d338-4c76-895d-81f76dc3ec77 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:20.516392 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823853c6-d338-4c76-895d-81f76dc3ec77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:20.516670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823853c6-d338-4c76-895d-81f76dc3ec77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:20.528097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823853c6-d338-4c76-895d-81f76dc3ec77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:20.529859 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823853c6-d338-4c76-895d-81f76dc3ec77 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:20.541709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1362/2731] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:20.754757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a43a70dc8e4ba88c079f1f21a20cde'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.755576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a43a70dc8e4ba88c079f1f21a20cde'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.768858 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a43a70dc8e4ba88c079f1f21a20cde'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.879920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a43a70dc8e4ba88c079f1f21a20cde'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.895734 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a43a70dc8e4ba88c079f1f21a20cde'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:20.909656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1370/2732] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:20 2022] DELETE /identity/v3/users/b7a43a70dc8e4ba88c079f1f21a20cde => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:20.918987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:20.918987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:20.918987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:20.958167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:20.960307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:21.061964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1363/2733] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:21.077020 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:21.078899 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:21.184623 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9698-a5af-43f7-9efd-a1281a4c1794 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:21.191380 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:21.195913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:21.195913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:21.195913 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:21.211288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:21.215005 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:21.235638 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1371/2734] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:21.368216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:21.371383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:21.392892 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b49c-7ed4-47c9-983c-b161a7f17b76 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:21.399524 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:21.400233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:21.400541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:21.401008 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] PATH_INFO: `/v3/projects/c18ce7a0f3f7423a847de666cfd4d5b6` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:21.402857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c18ce7a0f3f7423a847de666cfd4d5b6)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:21.405014 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:21.602446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18ce7a0f3f7423a847de666cfd4d5b6', 'request_id': 'req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:21.611367 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18ce7a0f3f7423a847de666cfd4d5b6', 'request_id': 'req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:21.746768 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18ce7a0f3f7423a847de666cfd4d5b6', 'request_id': 'req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:21.761634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] The token cache is being invalidate because project c18ce7a0f3f7423a847de666cfd4d5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:21.762320 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:21.763033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:21.764779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1364/2735] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:21 2022] DELETE /identity/v3/projects/c18ce7a0f3f7423a847de666cfd4d5b6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:21.769236 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:21.780984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ece8de-7281-46a2-9789-43c3ab634c3d service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:21.875313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:21.876470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:21.876934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:21.877298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] PATH_INFO: `/v3/users/4805c285aa344d9ca77663074045e622` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:21.878806 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4805c285aa344d9ca77663074045e622)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:21.881924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:22.060911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:22.067516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:22.081658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0143d4-68e2-48f4-aa86-d3b08ea5508b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:22.086747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:22.087676 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:22.088926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:22.089602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] PATH_INFO: `/v3/projects/923c682cd63f4c839f556d4bf26e8d27` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:22.091060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=923c682cd63f4c839f556d4bf26e8d27)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:22.092995 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:22.219086 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4805c285aa344d9ca77663074045e622'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.219830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4805c285aa344d9ca77663074045e622'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.237991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4805c285aa344d9ca77663074045e622'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.251127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:22.271720 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:22.275600 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923c682cd63f4c839f556d4bf26e8d27', 'request_id': 'req-6c272950-d035-4f28-b46a-87270d48adbe'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.282958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923c682cd63f4c839f556d4bf26e8d27', 'request_id': 'req-6c272950-d035-4f28-b46a-87270d48adbe'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.342068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4805c285aa344d9ca77663074045e622'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.351729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4805c285aa344d9ca77663074045e622'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.359111 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1372/2736] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:21 2022] DELETE /identity/v3/users/4805c285aa344d9ca77663074045e622 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:22.651322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923c682cd63f4c839f556d4bf26e8d27', 'request_id': 'req-6c272950-d035-4f28-b46a-87270d48adbe'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.664942 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] The token cache is being invalidate because project 923c682cd63f4c839f556d4bf26e8d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:22.665624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.666693 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:22.668662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1365/2737] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:22 2022] DELETE /identity/v3/projects/923c682cd63f4c839f556d4bf26e8d27 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:22.673145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:22.674848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:22.766405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f23b6-41dd-4202-b725-66e0a15bf12a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:22.771925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:22.776879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:22.776879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:22.776879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] PATH_INFO: `/v3/users/e12a822f333e4003ae001b193a36554a` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:22.776879 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12a822f333e4003ae001b193a36554a)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:22.778812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:23.117603 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:23.121173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:23.149275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12a822f333e4003ae001b193a36554a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.150151 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12a822f333e4003ae001b193a36554a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.171727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12a822f333e4003ae001b193a36554a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.220141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c272950-d035-4f28-b46a-87270d48adbe admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:23.225493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:23.226183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:23.226465 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:23.226743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] PATH_INFO: `/v3/projects/2f9d3976ff46464bb63c961f942d5184` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:23.228108 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9d3976ff46464bb63c961f942d5184)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:23.230706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:23.307408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12a822f333e4003ae001b193a36554a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.317174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f1eaccbb-273f-46ae-a64b-625a955a8c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12a822f333e4003ae001b193a36554a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.325264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1373/2738] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:22 2022] DELETE /identity/v3/users/e12a822f333e4003ae001b193a36554a => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:23.407602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9d3976ff46464bb63c961f942d5184', 'request_id': 'req-18133a93-eade-4023-9c75-7c150655b30a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.415841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9d3976ff46464bb63c961f942d5184', 'request_id': 'req-18133a93-eade-4023-9c75-7c150655b30a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.504330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9d3976ff46464bb63c961f942d5184', 'request_id': 'req-18133a93-eade-4023-9c75-7c150655b30a'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.522317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] The token cache is being invalidate because project 2f9d3976ff46464bb63c961f942d5184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:23.522806 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.523835 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-18133a93-eade-4023-9c75-7c150655b30a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:23.526298 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1366/2739] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:23 2022] DELETE /identity/v3/projects/2f9d3976ff46464bb63c961f942d5184 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:23.533835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:23.534336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:23.534778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:23.562948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:23.564229 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:23.663538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1374/2740] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:23.670189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:23.670348 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:23.671047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:23.698462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:23.701091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:23.713956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1367/2741] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:23.719792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:23.740455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538d080-70ce-4906-96bc-f246b285c544 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:23.749570 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:23.751089 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:23.751699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:23.752306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:23.753696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:23.754619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:23.757991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:23.784063 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1375/2742] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:15:23.788714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:23.799218 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea48199-b6e4-46ec-b0cf-08a7e79b0571 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:23.806719 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:23.807967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:23.808797 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:23.809311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:23.810572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:23.814669 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.029867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.032094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:24.048185 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fff91-3454-417e-b75b-74085ee4e6d5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.058558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.059576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.060041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.060523 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] PATH_INFO: `/v3/projects/b49b14278b2a484d967d522aa20ae1cf` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.070479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b49b14278b2a484d967d522aa20ae1cf)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.073183 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.104570 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1368/2743] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:23 2022] POST /identity/v3/projects => generated 389 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:24.108448 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.176985 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d55ec-4af1-441d-9a7d-6a8b6c3e1d5b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.181816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.182481 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.182760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.183033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.183674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.185629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.282584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b49b14278b2a484d967d522aa20ae1cf', 'request_id': 'req-21471bee-d547-48ac-a3a8-af072858232d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.291546 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b49b14278b2a484d967d522aa20ae1cf', 'request_id': 'req-21471bee-d547-48ac-a3a8-af072858232d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.302156 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b49b14278b2a484d967d522aa20ae1cf', 'request_id': 'req-21471bee-d547-48ac-a3a8-af072858232d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.347504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1369/2744] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:24 2022] POST /identity/v3/users => generated 295 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:24.352368 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.364167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c1480-0a70-4d29-be0e-ba77d31739e1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.442840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.442840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.442840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.442840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.443346 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.444081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.456616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1370/2745] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:24.458628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] The token cache is being invalidate because project b49b14278b2a484d967d522aa20ae1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:24.458925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.463125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.463125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1376/2746] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:24 2022] DELETE /identity/v3/projects/b49b14278b2a484d967d522aa20ae1cf => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:24.464525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.465849 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.466813 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:24.467329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:24.482203 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21471bee-d547-48ac-a3a8-af072858232d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.483935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f90c19-31b6-4c3e-926f-93d480a50825 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.487960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.488562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.488916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.489316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] PATH_INFO: `/v3/users/146d7433e9614b26b4c7a03d2ba19716` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.491140 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146d7433e9614b26b4c7a03d2ba19716)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.492830 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.497519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.497519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.497519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.497519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.498185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.500671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.519913 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1371/2747] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:24.523989 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.536386 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a05f-5cea-46bf-b357-b0c6e4bef322 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.541936 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92bc941-7127-479d-8ed2-c430d33b7e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.542634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bc941-7127-479d-8ed2-c430d33b7e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.542912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bc941-7127-479d-8ed2-c430d33b7e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.543183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bc941-7127-479d-8ed2-c430d33b7e4b admin admin] PATH_INFO: `/v3/projects/e9ead72f2e4a4dbab0b73c283654216a/users/0303933d1d8043ac859f6a30e60f7a43/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.560980 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bc941-7127-479d-8ed2-c430d33b7e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ead72f2e4a4dbab0b73c283654216a, user_id=0303933d1d8043ac859f6a30e60f7a43, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.563728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bc941-7127-479d-8ed2-c430d33b7e4b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:24.688826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1372/2748] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:24 2022] PUT /identity/v3/projects/e9ead72f2e4a4dbab0b73c283654216a/users/0303933d1d8043ac859f6a30e60f7a43/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:24.702125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.704068 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.704359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.746131 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] MFA Rules not processed for user `0303933d1d8043ac859f6a30e60f7a43`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:24.748288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:24.832662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1373/2749] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:24.838930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:24.856743 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146d7433e9614b26b4c7a03d2ba19716'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.857736 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146d7433e9614b26b4c7a03d2ba19716'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.877771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146d7433e9614b26b4c7a03d2ba19716'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:24.942097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f57afc-4b4c-461d-9c09-afcd25dc7ce2 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:24.950118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:24.951257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:24.951757 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:24.952299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:24.953467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:24.957529 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:25.050256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146d7433e9614b26b4c7a03d2ba19716'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:25.061024 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146d7433e9614b26b4c7a03d2ba19716'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:25.068705 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1377/2750] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:24 2022] DELETE /identity/v3/users/146d7433e9614b26b4c7a03d2ba19716 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:25.139817 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1374/2751] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:24 2022] POST /identity/v3/projects => generated 387 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:25.145079 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:25.148097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:25.268143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:25.276045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15d699-58c4-4282-8006-186285a82f90 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:25.370155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:25.371248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:25.371752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:25.372253 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:25.373469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:25.377747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:25.396822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:25.559278 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1378/2752] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:25 2022] POST /identity/v3/users => generated 300 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:25.561278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:25.578484 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b14d9-45bc-447d-b983-16eb8173d731 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:25.587561 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:25.589407 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:25.589916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:25.590449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:25.591903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:25.595376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:25.616146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1375/2753] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:25.623104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:25.642298 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11714683-4048-46a8-adc8-2767330dd6da admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:25.650159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:25.651903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:25.652415 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:25.652929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] PATH_INFO: `/v3/projects/1185bcb1beae46f3898793fbe091c896/users/13ccb1e06610463a9ab37b44127365c4/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:25.682277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1185bcb1beae46f3898793fbe091c896, user_id=13ccb1e06610463a9ab37b44127365c4, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:25.687761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:25.756739 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:25.759350 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:25.779432 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86600e-e3b3-4007-a35c-0c47237169f3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:25.788197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:25.789130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:25.789712 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:25.790157 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] PATH_INFO: `/v3/projects/265cc0fe67bd477ca53de249cd3f7917` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:25.794161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265cc0fe67bd477ca53de249cd3f7917)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:25.796743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:25.843445 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1379/2754] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:25 2022] PUT /identity/v3/projects/1185bcb1beae46f3898793fbe091c896/users/13ccb1e06610463a9ab37b44127365c4/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:25.847707 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:25.929809 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48322e0-139b-420d-81c3-afa8b69ab516 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:25.935407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:25.936083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:25.936406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:25.936720 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:25.937495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:25.939302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:25.955598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1380/2755] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:25.957488 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:25.971902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f81a42-e27c-4fe8-856c-519cd385e52c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:25.972816 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265cc0fe67bd477ca53de249cd3f7917', 'request_id': 'req-668e094d-72dc-4fa6-851a-e26da2b2d9b5'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:25.982369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265cc0fe67bd477ca53de249cd3f7917', 'request_id': 'req-668e094d-72dc-4fa6-851a-e26da2b2d9b5'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.043121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17d8ac-49df-4b87-a719-b2942b088657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.043718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17d8ac-49df-4b87-a719-b2942b088657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.044011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17d8ac-49df-4b87-a719-b2942b088657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.044296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17d8ac-49df-4b87-a719-b2942b088657 admin admin] PATH_INFO: `/v3/projects/1185bcb1beae46f3898793fbe091c896/users/13ccb1e06610463a9ab37b44127365c4/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.046426 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17d8ac-49df-4b87-a719-b2942b088657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1185bcb1beae46f3898793fbe091c896, user_id=13ccb1e06610463a9ab37b44127365c4, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.049135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17d8ac-49df-4b87-a719-b2942b088657 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.062722 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1381/2756] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:25 2022] PUT /identity/v3/projects/1185bcb1beae46f3898793fbe091c896/users/13ccb1e06610463a9ab37b44127365c4/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:26.074852 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.075144 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.075427 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.078774 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265cc0fe67bd477ca53de249cd3f7917', 'request_id': 'req-668e094d-72dc-4fa6-851a-e26da2b2d9b5'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.095240 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] The token cache is being invalidate because project 265cc0fe67bd477ca53de249cd3f7917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:26.095755 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.096993 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-668e094d-72dc-4fa6-851a-e26da2b2d9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.099715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1376/2757] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:25 2022] DELETE /identity/v3/projects/265cc0fe67bd477ca53de249cd3f7917 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:26.105358 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe324c9-d5eb-4da7-af5e-11dd09d76eed None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.105826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe324c9-d5eb-4da7-af5e-11dd09d76eed None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.106328 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe324c9-d5eb-4da7-af5e-11dd09d76eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.118631 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] MFA Rules not processed for user `13ccb1e06610463a9ab37b44127365c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:26.119927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:26.128679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-bbe324c9-d5eb-4da7-af5e-11dd09d76eed None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:26.129987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bbe324c9-d5eb-4da7-af5e-11dd09d76eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:26.201605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1377/2758] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:26.210219 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.216955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.216955 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.217293 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1382/2759] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:26.226950 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.228580 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:26.242245 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:26.244438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:26.257922 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1378/2760] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:26.264479 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.279784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931ed69-f473-41e7-a5b0-127a1ceb2500 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.286741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.287553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.287883 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.288165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.288862 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:26.289210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.291207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.314159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1379/2761] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:15:26.314417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8acb2-7d8d-4cbd-9f3f-8788374a7aeb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.316440 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.317996 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.318956 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.319340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.319792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.333234 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.334808 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.335089 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0f3d7-3904-4de6-944d-e9aa0292470b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.343132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.344129 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.344584 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.345035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.346207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.346973 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1383/2762] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:26.350890 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.534920 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.536614 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:26.552194 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb6e1-bda2-4f69-8c97-31e93c3993ae service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.557697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.558394 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.558680 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.558963 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] PATH_INFO: `/v3/users/0303933d1d8043ac859f6a30e60f7a43` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.560242 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0303933d1d8043ac859f6a30e60f7a43)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.562279 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.626250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1380/2763] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:26 2022] POST /identity/v3/projects => generated 371 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:26.630963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.704843 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1aec46-0586-487b-befc-d87559e09997 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.712612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.713683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.714189 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.714714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.715602 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.719128 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.922125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1381/2764] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:26 2022] POST /identity/v3/users => generated 286 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:26.928981 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.932895 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0303933d1d8043ac859f6a30e60f7a43'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.934889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0303933d1d8043ac859f6a30e60f7a43'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.946141 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3571-6117-41ca-91f0-38414883554c admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.951890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0303933d1d8043ac859f6a30e60f7a43'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:26.952781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.953756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.954252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.954724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.955825 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.958316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:26.970550 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1382/2765] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:26.971438 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:26.973176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:26.987754 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f5e05-3f22-4ddf-b968-8c35ea7231e2 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:26.994601 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:26.995251 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:26.995538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:26.995849 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] PATH_INFO: `/v3/users/0854939703534ebb9cf20474167ce68b` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:26.997376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0854939703534ebb9cf20474167ce68b)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:26.999442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:27.064022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0303933d1d8043ac859f6a30e60f7a43'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.072431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0303933d1d8043ac859f6a30e60f7a43'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.079166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1384/2766] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:26 2022] DELETE /identity/v3/users/0303933d1d8043ac859f6a30e60f7a43 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:27.080068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:27.081697 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.158655 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54584a28-cd7c-42fe-8ea9-7066d6931e07 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:27.167582 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178f00c-fb6e-4d15-92fe-3cdc7f58263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:27.168544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178f00c-fb6e-4d15-92fe-3cdc7f58263f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.168953 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178f00c-fb6e-4d15-92fe-3cdc7f58263f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.169337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178f00c-fb6e-4d15-92fe-3cdc7f58263f admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.170382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178f00c-fb6e-4d15-92fe-3cdc7f58263f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:27.173142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178f00c-fb6e-4d15-92fe-3cdc7f58263f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:27.193535 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1385/2767] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:27.195867 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.196350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.196813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.231807 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:27.234043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.250935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1386/2768] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:27.250935 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:27.262150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:27.269250 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2cab5-84bb-46fd-b9b5-d3a4847f87c2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:27.278402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:27.279530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.280004 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.280498 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] PATH_INFO: `/v3/projects/493732d3c282440f8593558858bb0b33/users/176ac62de5974fef91b59293df718ebc/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.308475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493732d3c282440f8593558858bb0b33, user_id=176ac62de5974fef91b59293df718ebc, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:27.314065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:27.359978 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0854939703534ebb9cf20474167ce68b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.538805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1387/2769] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:27 2022] PUT /identity/v3/projects/493732d3c282440f8593558858bb0b33/users/176ac62de5974fef91b59293df718ebc/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:27.539686 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:27.541519 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.599559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0854939703534ebb9cf20474167ce68b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.603259 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:27.624068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c1831-426e-4bd1-9ce0-9f990116da71 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:27.626301 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0854939703534ebb9cf20474167ce68b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.627249 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0854939703534ebb9cf20474167ce68b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.629489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850bf83-8fab-4e22-bde2-524233690135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:27.630444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850bf83-8fab-4e22-bde2-524233690135 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.630922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850bf83-8fab-4e22-bde2-524233690135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.631362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850bf83-8fab-4e22-bde2-524233690135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.639469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-53aff070-9d54-4e29-9480-8bdb8a1f676f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0854939703534ebb9cf20474167ce68b'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:27.651239 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1383/2770] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:26 2022] DELETE /identity/v3/users/0854939703534ebb9cf20474167ce68b => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:27.653965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.654469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.655153 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.684826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:27.686847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.730341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850bf83-8fab-4e22-bde2-524233690135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:27.732137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850bf83-8fab-4e22-bde2-524233690135 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:27.745562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1388/2771] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:27.746946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.747277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.747513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.772578 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:27.773932 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.777117 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1384/2772] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:27.785177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1389/2773] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:27.792167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:27.795525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.808013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:27.811133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:27.910831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68f0f4-a4db-46a5-be94-0d6d2f9198c2 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:27.915138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a70f91-119e-43e7-94f2-b665ee6baf0f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:27.918737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:27.919861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.920542 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.921366 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.922460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:27.923475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:27.923983 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:27.924476 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:27.933957 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:27.935539 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:27.943428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:27.946345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:27.946854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1385/2774] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:27.971113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1390/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:28.298827 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:28.429152 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:28.544254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:28.544254 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:28.555749 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f7e9-42d8-428e-b127-b8c0c94978fc service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:28.561930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:28.562592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:28.562874 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:28.563152 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] PATH_INFO: `/v3/projects/e9ead72f2e4a4dbab0b73c283654216a` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:28.564333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ead72f2e4a4dbab0b73c283654216a)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:28.566409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:28.645530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:28.647029 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:28.661528 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d140179-ae69-4016-a7cf-7bb245e890e2 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:28.670106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:28.671053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:28.671529 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:28.671977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] PATH_INFO: `/v3/projects/966da54c52d64018a7a84f0f45bebb73` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:28.673350 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966da54c52d64018a7a84f0f45bebb73)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:28.675565 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:28.746504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ead72f2e4a4dbab0b73c283654216a', 'request_id': 'req-8081d314-c7e9-4663-82b4-e54a2b7a33df'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.752974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ead72f2e4a4dbab0b73c283654216a', 'request_id': 'req-8081d314-c7e9-4663-82b4-e54a2b7a33df'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.945398 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966da54c52d64018a7a84f0f45bebb73', 'request_id': 'req-0cf7db98-d773-40f6-b016-834cc2c26602'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.948591 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ead72f2e4a4dbab0b73c283654216a', 'request_id': 'req-8081d314-c7e9-4663-82b4-e54a2b7a33df'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.953691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966da54c52d64018a7a84f0f45bebb73', 'request_id': 'req-0cf7db98-d773-40f6-b016-834cc2c26602'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.960642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966da54c52d64018a7a84f0f45bebb73', 'request_id': 'req-0cf7db98-d773-40f6-b016-834cc2c26602'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.965721 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] The token cache is being invalidate because project e9ead72f2e4a4dbab0b73c283654216a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:28.966271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.967631 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:28.973801 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1386/2776] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:28 2022] DELETE /identity/v3/projects/e9ead72f2e4a4dbab0b73c283654216a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:28.975331 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:28.976857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:29.051246 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081d314-c7e9-4663-82b4-e54a2b7a33df admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:29.121278 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:29.121950 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:29.122238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:29.122512 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] PATH_INFO: `/v3/users/13ccb1e06610463a9ab37b44127365c4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:29.123786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ccb1e06610463a9ab37b44127365c4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:29.125763 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:29.144329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] The token cache is being invalidate because project 966da54c52d64018a7a84f0f45bebb73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:29.145225 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.147354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.152107 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1391/2777] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:28 2022] DELETE /identity/v3/projects/966da54c52d64018a7a84f0f45bebb73 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:29.156836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:29.159669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:29.187635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7db98-d773-40f6-b016-834cc2c26602 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:29.192386 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:29.193116 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:29.193406 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:29.193814 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] PATH_INFO: `/v3/users/5444dbf072e34b29aad12ee26cbd5efa` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:29.195252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5444dbf072e34b29aad12ee26cbd5efa)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:29.198173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:29.476831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ccb1e06610463a9ab37b44127365c4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.591568 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5444dbf072e34b29aad12ee26cbd5efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.592263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5444dbf072e34b29aad12ee26cbd5efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.614664 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5444dbf072e34b29aad12ee26cbd5efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.632880 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ccb1e06610463a9ab37b44127365c4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.649565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ccb1e06610463a9ab37b44127365c4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.651066 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ccb1e06610463a9ab37b44127365c4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.660204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7203696d-69e8-4f0a-baf8-8431cb4c9438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ccb1e06610463a9ab37b44127365c4'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.667056 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1387/2778] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:28 2022] DELETE /identity/v3/users/13ccb1e06610463a9ab37b44127365c4 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:29.767726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5444dbf072e34b29aad12ee26cbd5efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.785392 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5444dbf072e34b29aad12ee26cbd5efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:29.794676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1392/2779] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:29 2022] DELETE /identity/v3/users/5444dbf072e34b29aad12ee26cbd5efa => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:29.812413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:30.340714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:30.341681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:30.342171 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:30.361548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:30.382597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] MFA Rules not processed for user `176ac62de5974fef91b59293df718ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:30.383920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:30.493237 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1388/2780] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:30.508783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:30.512145 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:30.638923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053d685-e934-4099-b969-c135dce37ec8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:30.645709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:30.647588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:30.648448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:30.649741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:30.669239 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:30.672264 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:30.691425 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1393/2781] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:30.814399 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:30.816789 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:30.867481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:30.869669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:30.959443 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225bec2-caa9-4176-bff0-286aff295cef None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:30.964598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:30.965332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:30.965617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:30.965923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] PATH_INFO: `/v3/projects/1185bcb1beae46f3898793fbe091c896` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:30.967151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1185bcb1beae46f3898793fbe091c896)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:30.969092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:30.975813 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557d772-3cc0-4b27-af30-a75a6decc809 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:30.983137 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:30.984067 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:30.984477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:30.984872 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] PATH_INFO: `/v3/projects/2b96f0958fd14ee2ac9b1318ead77017` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:30.986654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b96f0958fd14ee2ac9b1318ead77017)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:30.989832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:31.140060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1185bcb1beae46f3898793fbe091c896', 'request_id': 'req-515fde57-8a3d-483c-b739-94d2718e07f6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.147262 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1185bcb1beae46f3898793fbe091c896', 'request_id': 'req-515fde57-8a3d-483c-b739-94d2718e07f6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.195452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b96f0958fd14ee2ac9b1318ead77017', 'request_id': 'req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.203811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b96f0958fd14ee2ac9b1318ead77017', 'request_id': 'req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.211043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b96f0958fd14ee2ac9b1318ead77017', 'request_id': 'req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.279031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1185bcb1beae46f3898793fbe091c896', 'request_id': 'req-515fde57-8a3d-483c-b739-94d2718e07f6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.292744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] The token cache is being invalidate because project 1185bcb1beae46f3898793fbe091c896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:31.293049 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.293891 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-515fde57-8a3d-483c-b739-94d2718e07f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.295514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1389/2782] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:30 2022] DELETE /identity/v3/projects/1185bcb1beae46f3898793fbe091c896 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:31.300984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99c86-9b24-4e09-916b-87e1abcde7d0 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.301279 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99c86-9b24-4e09-916b-87e1abcde7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.301530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99c86-9b24-4e09-916b-87e1abcde7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.326467 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-3fc99c86-9b24-4e09-916b-87e1abcde7d0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:31.327821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-3fc99c86-9b24-4e09-916b-87e1abcde7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:31.382345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] The token cache is being invalidate because project 2b96f0958fd14ee2ac9b1318ead77017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:31.382744 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.383604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-400e36e4-ecbe-4ae1-bbd7-e639bfe30a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:31.386472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1394/2783] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:30 2022] DELETE /identity/v3/projects/2b96f0958fd14ee2ac9b1318ead77017 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:31.393082 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8572de-4556-407d-9733-cf204e0467a9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.393428 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8572de-4556-407d-9733-cf204e0467a9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.393781 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8572de-4556-407d-9733-cf204e0467a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.407579 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1390/2784] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:31.416446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ea8572de-4556-407d-9733-cf204e0467a9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:31.417285 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.417576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.417991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.418155 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ea8572de-4556-407d-9733-cf204e0467a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:31.426527 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1395/2785] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:31.434675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.435142 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.435252 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.442236 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:31.443756 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:31.454651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1391/2786] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:31.457711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:31.458988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:31.459727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:31.469191 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1396/2787] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:31.471205 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a77d7-93dd-4c82-a081-043710d353d2 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:31.474820 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:31.476651 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:31.477299 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.477577 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.477853 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.478604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:31.478910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:31.480566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:31.485718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c9a76-29bd-424b-9d94-da4b36037052 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:31.491030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:31.491660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.491944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.492251 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.492918 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:31.493228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:31.498166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1392/2788] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:15:31.498656 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:31.499057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:31.510656 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f817e-088c-42f1-a7e0-31cc6b2116ed admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:31.510992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1397/2789] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:15:31.518751 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:31.519401 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.519737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.520022 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.520673 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:31.522704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:31.524125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:31.543478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cd63-09c6-4f35-a34e-11180ef2e5b3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:31.549069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:31.553835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.553835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.553835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.553835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:31.553835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:31.697237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1398/2791] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:31.697526 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1393/2791] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:31.701998 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:31.702153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:31.794124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9094-8f13-4305-bb34-fd74292f9f95 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:31.796704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2722f-a113-4206-8549-d1ecccd93d60 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:31.801326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:31.801982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.802273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.802555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.803211 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:31.803330 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:31.804426 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:31.804944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:31.805417 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:31.805559 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:31.807238 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:31.811317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.008040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1399/2792] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/users => generated 292 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:32.012919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.025030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d932bd-5ffd-4179-b9de-f632295c2591 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.030751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.031431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.031782 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.032129 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.032845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.034721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.047041 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1400/2793] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:32.051979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.065453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1394/2794] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:31 2022] POST /identity/v3/users => generated 290 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:32.066881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f0b8e-cad9-43e4-909f-aa8a366cc6b7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.072810 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.075458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.076021 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.076459 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.077088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.078170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.080757 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.088911 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66973964-4139-4a56-b577-c8b7ab0da5a9 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.094975 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.096527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.096527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.096527 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.097137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.098432 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1401/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:32.099257 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.103033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.113457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1395/2796] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:32.115160 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a8f5a-6bcc-4c47-92d9-cb0c25e6e81c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.118490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.121362 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091c292-6705-4607-9cda-8c62ffd7f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.122104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091c292-6705-4607-9cda-8c62ffd7f9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.122388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091c292-6705-4607-9cda-8c62ffd7f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.122711 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091c292-6705-4607-9cda-8c62ffd7f9ca admin admin] PATH_INFO: `/v3/projects/b341473edf974333a51541884f3cabd7/users/9169594715b54a1bbe347df790fb20d9/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.130587 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bd0e0-c03a-4a0c-a476-db7e4c544f8b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.136549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.137186 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.137469 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.137747 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.138442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.140263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.142803 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091c292-6705-4607-9cda-8c62ffd7f9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b341473edf974333a51541884f3cabd7, user_id=9169594715b54a1bbe347df790fb20d9, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.148569 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091c292-6705-4607-9cda-8c62ffd7f9ca admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.162457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1396/2797] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:32.168268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.179552 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c03b-6913-4916-96af-84db1fb91b7d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.185760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85d414-944f-40d7-8b8c-6bde97228a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.187910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85d414-944f-40d7-8b8c-6bde97228a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.187910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85d414-944f-40d7-8b8c-6bde97228a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.187910 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85d414-944f-40d7-8b8c-6bde97228a82 admin admin] PATH_INFO: `/v3/projects/35527d377e78434aba3d2cbb6a0ba0b1/users/ae04cb2987f54b9589f4e997eec3de7e/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.205865 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85d414-944f-40d7-8b8c-6bde97228a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35527d377e78434aba3d2cbb6a0ba0b1, user_id=ae04cb2987f54b9589f4e997eec3de7e, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.208838 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85d414-944f-40d7-8b8c-6bde97228a82 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.273050 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1402/2798] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:32 2022] PUT /identity/v3/projects/b341473edf974333a51541884f3cabd7/users/9169594715b54a1bbe347df790fb20d9/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:32.285169 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.285517 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.285721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.321661 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1397/2799] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:32 2022] PUT /identity/v3/projects/35527d377e78434aba3d2cbb6a0ba0b1/users/ae04cb2987f54b9589f4e997eec3de7e/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:32.327663 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:32.331374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] MFA Rules not processed for user `9169594715b54a1bbe347df790fb20d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:32.332640 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:32.341655 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.342174 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.342692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.389380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] MFA Rules not processed for user `ae04cb2987f54b9589f4e997eec3de7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:32.391857 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:32.410489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1403/2800] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:32.415447 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.497553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1398/2801] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:32.502831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.508158 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7fbd0-3757-4f9a-9e03-e6f49e05dc48 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.517709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7f7b0-c81a-4e40-b8c1-fbcd2ea40634 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.518130 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.518896 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.519247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.519742 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.521070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.523560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.524195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.524480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.524760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.524906 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.526083 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.528093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.681502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1399/2802] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:32.681854 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1404/2803] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:32.682037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.682929 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:32.773439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaba187-7457-408f-a025-6c46eeb84f69 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.773930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4c9f-abc1-4b2b-ba6d-b5e26de3532b admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:32.774040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e80190-a37a-4831-88be-163557a83496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.774040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e80190-a37a-4831-88be-163557a83496 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.774040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e80190-a37a-4831-88be-163557a83496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.774040 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e80190-a37a-4831-88be-163557a83496 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.774471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:32.774579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e80190-a37a-4831-88be-163557a83496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.774681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.774681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.774681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.774681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:32.774681 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.776714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e80190-a37a-4831-88be-163557a83496 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:32.809811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:32.919128 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1405/2804] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:32.920617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:32.920904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:32.921174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:32.943408 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] MFA Rules not processed for user `176ac62de5974fef91b59293df718ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:32.944698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:33.018992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1400/2805] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/users => generated 297 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:33.018992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.027053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1406/2806] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:33.028579 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.045984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd940-cf4c-4613-be35-3373180545e3 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.050344 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ea4ae-a537-4fe2-b4c4-f4aa04ef66b0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.054670 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.055854 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.056897 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.057565 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.058798 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.061125 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.062175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.062369 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.062792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.063255 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.064466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.067553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.088905 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1401/2807] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:33.094081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1407/2808] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:33.094324 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.096777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.102637 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2e4d6-8685-4823-9944-02c5e9a1b032 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.109252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.109988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.110271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.110572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] PATH_INFO: `/v3/projects/6c77841b624e49b5abbe493a749f2e4c/users/511432be94614beba342d02bca9385b6/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.119714 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4654e2-0ab0-43bc-b616-2c1e15b0cbc1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.126591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.127700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.127700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.128060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] PATH_INFO: `/v3/projects/93b5636236104c40b6db2ddad3e64a8b/users/9787a7181f364357bc259124b7d6a827/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.140221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c77841b624e49b5abbe493a749f2e4c, user_id=511432be94614beba342d02bca9385b6, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.140221 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.155109 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b5636236104c40b6db2ddad3e64a8b, user_id=9787a7181f364357bc259124b7d6a827, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.161276 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.316144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1402/2809] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:33 2022] PUT /identity/v3/projects/6c77841b624e49b5abbe493a749f2e4c/users/511432be94614beba342d02bca9385b6/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:33.323601 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.356944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1408/2810] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:33 2022] PUT /identity/v3/projects/93b5636236104c40b6db2ddad3e64a8b/users/9787a7181f364357bc259124b7d6a827/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:33.362876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.455450 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7aec-3277-49c1-b900-edde608826a0 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.480074 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9ac37-5849-4c26-b379-73655bc58f5a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.488624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.488942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.489466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.490066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.491894 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.494812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.510704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1409/2811] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:33.517402 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.538405 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd1632-86fe-47f0-95f1-887d4bf58265 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.544812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e909f48-0d93-4169-a6e4-c1507c2a0bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.545741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909f48-0d93-4169-a6e4-c1507c2a0bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.546260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909f48-0d93-4169-a6e4-c1507c2a0bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.546746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909f48-0d93-4169-a6e4-c1507c2a0bdd admin admin] PATH_INFO: `/v3/projects/93b5636236104c40b6db2ddad3e64a8b/users/9787a7181f364357bc259124b7d6a827/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.552717 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e909f48-0d93-4169-a6e4-c1507c2a0bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b5636236104c40b6db2ddad3e64a8b, user_id=9787a7181f364357bc259124b7d6a827, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.555064 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.556294 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.556770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.557297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.558262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e909f48-0d93-4169-a6e4-c1507c2a0bdd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.558572 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.561920 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.576832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1403/2812] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:33.583055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.589876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1410/2813] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:33 2022] PUT /identity/v3/projects/93b5636236104c40b6db2ddad3e64a8b/users/9787a7181f364357bc259124b7d6a827/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:33.604577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.605049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.605771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.608332 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf42d-1e31-4b9f-9806-7e7a56673f01 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.661800 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] MFA Rules not processed for user `9787a7181f364357bc259124b7d6a827`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:33.663847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:33.701452 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7fa4d-28e4-42db-868d-1021bd1aa97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.702498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7fa4d-28e4-42db-868d-1021bd1aa97c admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.703007 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7fa4d-28e4-42db-868d-1021bd1aa97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.703435 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7fa4d-28e4-42db-868d-1021bd1aa97c admin admin] PATH_INFO: `/v3/projects/6c77841b624e49b5abbe493a749f2e4c/users/511432be94614beba342d02bca9385b6/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.706662 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7fa4d-28e4-42db-868d-1021bd1aa97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c77841b624e49b5abbe493a749f2e4c, user_id=511432be94614beba342d02bca9385b6, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:33.710898 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7fa4d-28e4-42db-868d-1021bd1aa97c admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:33.737701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1404/2814] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:33 2022] PUT /identity/v3/projects/6c77841b624e49b5abbe493a749f2e4c/users/511432be94614beba342d02bca9385b6/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:33.757045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.760781 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.761155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:33.798316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1411/2815] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:33.811238 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.812942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:33.829288 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] MFA Rules not processed for user `511432be94614beba342d02bca9385b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:33.831748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:33.896979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9580366-a753-4d10-8d93-e326c2792058 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:33.938077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1405/2816] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:33.957442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:33.957442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:33.966758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:33.967413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:33.967693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:33.967964 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:34.036334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbb5c0-7a2f-4971-be6c-d1eccb93747b None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:34.042605 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:34.043566 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:34.044057 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:34.044671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:34.046710 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:34.065209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:34.065209 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:34.075724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:34.077220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:34.077672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1406/2817] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:34.089295 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1412/2818] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:34.253052 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:34.329402 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:34.342971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb9c6c-3365-4999-8f9e-68144b787320 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:34.350312 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:34.351571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:34.352065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:34.352541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:34.454213 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:34.456557 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:34.469214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1407/2819] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:34.863477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:34.935217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7cb5c-58be-429e-bff2-76dd821b14fd service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:34.940269 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:34.941014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:34.941296 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:34.941571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] PATH_INFO: `/v3/users/ae04cb2987f54b9589f4e997eec3de7e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:34.942790 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae04cb2987f54b9589f4e997eec3de7e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:34.944728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:35.015670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:35.216088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:35.287490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:35.305730 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fc34-b841-469b-895b-139e3661b527 service cinder] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:35.315124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:35.316388 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:35.316551 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:35.316861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] PATH_INFO: `/v3/users/9169594715b54a1bbe347df790fb20d9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:35.318991 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9169594715b54a1bbe347df790fb20d9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:35.321648 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:35.334131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae04cb2987f54b9589f4e997eec3de7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.336205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae04cb2987f54b9589f4e997eec3de7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.356326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae04cb2987f54b9589f4e997eec3de7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.483682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae04cb2987f54b9589f4e997eec3de7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.493321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-83eea66e-364d-44f8-bbeb-1a6104824962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae04cb2987f54b9589f4e997eec3de7e'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.501452 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1413/2820] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:34 2022] DELETE /identity/v3/users/ae04cb2987f54b9589f4e997eec3de7e => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:35.506704 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:35.507060 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:35.507379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:35.529984 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:35.538034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:35.616634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1414/2821] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:35.628532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:35.630123 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:35.671031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9169594715b54a1bbe347df790fb20d9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.701201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb9ed4-1434-41b9-9c82-3ce6991676ba None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:35.705448 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02a275-9c3a-4fb9-b5f9-947843e0def9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:35.706138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02a275-9c3a-4fb9-b5f9-947843e0def9 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:35.706421 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02a275-9c3a-4fb9-b5f9-947843e0def9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:35.706698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02a275-9c3a-4fb9-b5f9-947843e0def9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:35.717378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02a275-9c3a-4fb9-b5f9-947843e0def9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:35.719170 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02a275-9c3a-4fb9-b5f9-947843e0def9 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:35.731015 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1415/2822] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:15:35.787415 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9169594715b54a1bbe347df790fb20d9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.805784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9169594715b54a1bbe347df790fb20d9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.806946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9169594715b54a1bbe347df790fb20d9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.816812 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9169594715b54a1bbe347df790fb20d9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:35.825147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1408/2823] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:35 2022] DELETE /identity/v3/users/9169594715b54a1bbe347df790fb20d9 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:35.830254 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:35.830555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:35.830831 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:35.852794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:35.854138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:35.941556 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:35.944205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1416/2824] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:35.956537 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:35.958190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:36.029316 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b196b6-5b3e-44f5-90be-0509cc4c6204 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:36.033210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:36.033926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:36.034337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:36.034616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:36.050692 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:36.052190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:36.069609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1409/2825] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:36.132823 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:36.423741 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:36.436300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:36.439540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:36.472626 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695b7b-ed87-489c-b596-2d65318f13a6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:36.492032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:36.493180 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:36.494758 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:36.495446 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] PATH_INFO: `/v3/projects/35527d377e78434aba3d2cbb6a0ba0b1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:36.500450 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35527d377e78434aba3d2cbb6a0ba0b1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:36.504353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:36.513060 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:36.892193 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35527d377e78434aba3d2cbb6a0ba0b1', 'request_id': 'req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:36.899642 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35527d377e78434aba3d2cbb6a0ba0b1', 'request_id': 'req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:36.908477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35527d377e78434aba3d2cbb6a0ba0b1', 'request_id': 'req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:36.918307 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:36.918965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:36.939741 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393458-7336-4f4a-968b-7a4decf7ad5d service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:37.036471 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:37.037190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:37.037476 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:37.037752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] PATH_INFO: `/v3/projects/b341473edf974333a51541884f3cabd7` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:37.039111 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b341473edf974333a51541884f3cabd7)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:37.041166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:37.052948 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] The token cache is being invalidate because project 35527d377e78434aba3d2cbb6a0ba0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:37.053721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.054811 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.057778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1417/2826] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:36 2022] DELETE /identity/v3/projects/35527d377e78434aba3d2cbb6a0ba0b1 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:37.062265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:37.063926 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:37.080201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b72b0a-90ab-4b43-9bb7-7a033ced1e5d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:37.085619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:37.086327 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:37.086619 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:37.086902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] PATH_INFO: `/v3/users/9787a7181f364357bc259124b7d6a827` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:37.089979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9787a7181f364357bc259124b7d6a827)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:37.097605 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:37.201250 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b341473edf974333a51541884f3cabd7', 'request_id': 'req-8c6c7c35-75cd-4254-982d-9487a3268171'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.208697 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b341473edf974333a51541884f3cabd7', 'request_id': 'req-8c6c7c35-75cd-4254-982d-9487a3268171'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.352728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b341473edf974333a51541884f3cabd7', 'request_id': 'req-8c6c7c35-75cd-4254-982d-9487a3268171'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.354171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:37.366641 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] The token cache is being invalidate because project b341473edf974333a51541884f3cabd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:37.366946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.367617 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.369923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1410/2827] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:36 2022] DELETE /identity/v3/projects/b341473edf974333a51541884f3cabd7 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:37.374546 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:37.376053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:37.401934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9787a7181f364357bc259124b7d6a827'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.404341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9787a7181f364357bc259124b7d6a827'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.431966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9787a7181f364357bc259124b7d6a827'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.464183 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:37.467271 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7c35-75cd-4254-982d-9487a3268171 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:37.472916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:37.473195 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:37.473462 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:37.473744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] PATH_INFO: `/v3/users/511432be94614beba342d02bca9385b6` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:37.475033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=511432be94614beba342d02bca9385b6)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:37.477062 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:37.584634 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9787a7181f364357bc259124b7d6a827'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.597514 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9787a7181f364357bc259124b7d6a827'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.611490 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1418/2828] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:37 2022] DELETE /identity/v3/users/9787a7181f364357bc259124b7d6a827 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:37.842773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '511432be94614beba342d02bca9385b6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:37.970497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:38.056341 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '511432be94614beba342d02bca9385b6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.086793 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '511432be94614beba342d02bca9385b6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.087711 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '511432be94614beba342d02bca9385b6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.099390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '511432be94614beba342d02bca9385b6'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.111597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1411/2829] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:37 2022] DELETE /identity/v3/users/511432be94614beba342d02bca9385b6 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:38.267954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:38.311544 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:38.314031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:38.415205 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e3fb8-0049-48df-ae5f-28d606f81457 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:38.512313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:38.512313 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:38.512840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:38.513263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] PATH_INFO: `/v3/projects/93b5636236104c40b6db2ddad3e64a8b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:38.515727 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b5636236104c40b6db2ddad3e64a8b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:38.519094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:38.652870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b5636236104c40b6db2ddad3e64a8b', 'request_id': 'req-6043fa00-c758-42e2-8a72-0a72f44490de'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.661991 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b5636236104c40b6db2ddad3e64a8b', 'request_id': 'req-6043fa00-c758-42e2-8a72-0a72f44490de'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.669455 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b5636236104c40b6db2ddad3e64a8b', 'request_id': 'req-6043fa00-c758-42e2-8a72-0a72f44490de'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.774160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:38.778334 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:38.779146 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] The token cache is being invalidate because project 93b5636236104c40b6db2ddad3e64a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:38.779903 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.781721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6043fa00-c758-42e2-8a72-0a72f44490de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:38.785669 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1419/2830] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:38 2022] DELETE /identity/v3/projects/93b5636236104c40b6db2ddad3e64a8b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:38.796819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:38.797466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:38.798161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:38.813105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604a40-f0d0-4b52-9bf6-ca3dc45ea7dd admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:38.839549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:38.842078 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:38.868769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:38.930682 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:38.933017 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:38.933453 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:38.933946 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] PATH_INFO: `/v3/projects/6c77841b624e49b5abbe493a749f2e4c` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:38.936118 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c77841b624e49b5abbe493a749f2e4c)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:38.936960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1420/2831] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:38.937635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:38.939069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:38.941144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.023167 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3342df-0c6e-4ca7-862d-50d1a4d56bc4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.031495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.032540 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.033032 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.033516 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.036716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:39.114642 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c77841b624e49b5abbe493a749f2e4c', 'request_id': 'req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:39.125758 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c77841b624e49b5abbe493a749f2e4c', 'request_id': 'req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:39.148752 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.151330 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5727a-8908-49b2-9110-ce69797895c5 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.163153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1421/2832] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:39.164489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.164780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.165057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.192103 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:39.197022 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:39.251343 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c77841b624e49b5abbe493a749f2e4c', 'request_id': 'req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:39.267752 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] The token cache is being invalidate because project 6c77841b624e49b5abbe493a749f2e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:39.268707 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:39.269808 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-8c963caf-14b7-48e8-b88a-9a3b5acb08c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:39.272344 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1412/2833] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:38 2022] DELETE /identity/v3/projects/6c77841b624e49b5abbe493a749f2e4c => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:39.279076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e85a-ab42-4a97-81f0-c1802b11e2b7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.279701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e85a-ab42-4a97-81f0-c1802b11e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.280137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e85a-ab42-4a97-81f0-c1802b11e2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.306859 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1422/2834] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:39.310851 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.319727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2056e85a-ab42-4a97-81f0-c1802b11e2b7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:39.320685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4ecde-772a-4a63-b15f-71e003aaadb9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.324046 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2056e85a-ab42-4a97-81f0-c1802b11e2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:39.328045 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.328687 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.328972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.329363 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.330181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:39.330692 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.332334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.347166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1423/2835] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:15:39.351772 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.355917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1413/2836] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:39.363764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.364201 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.364710 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.370043 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825076a-d6d6-41b0-9ec8-9f74547d21b1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.375695 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.376337 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.376766 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.378066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.378066 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.379871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.390840 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:39.392191 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:39.402497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1414/2837] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:39.406805 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.417873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0320d1-c1b8-4804-9262-432454c6fae7 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.423182 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.423814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.424098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.424374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.425082 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:15:39.425449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.427151 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.441377 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1415/2838] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:15:39.446436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.460657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb1fc7-d7c5-4a51-99e5-f2b2af6e2e4e admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.467506 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.468403 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.468785 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.469163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.470310 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.472071 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.495812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1424/2839] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:39.500690 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.587530 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300894f-0b59-4915-adcc-62731609f8e9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.591021 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1416/2840] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:39.595257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.596966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.596966 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.597361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.597521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.598312 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.602273 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.686604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9876a9-e282-433c-b0e7-1e68dec7332d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.694490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.695229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.695663 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.696198 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.697159 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.700065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.723927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1425/2841] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:39.729739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.741263 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98ac05-9f23-46cb-819d-371e28b2bdd3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.747135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.747762 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.748100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.748379 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.749034 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.750784 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.762486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1426/2842] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:39.766756 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.778359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d7f7d-5282-4adf-920e-a9ef4f834fae admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.783793 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.788416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.788416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.788416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.788416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.788416 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.802359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1427/2843] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:39.807105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.818098 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d22f8-c144-4408-bd11-ed5b0efb43e0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.823819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69b6a3-9835-4c6f-9eff-89ae273c9110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.824660 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69b6a3-9835-4c6f-9eff-89ae273c9110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.825053 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69b6a3-9835-4c6f-9eff-89ae273c9110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.825495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69b6a3-9835-4c6f-9eff-89ae273c9110 admin admin] PATH_INFO: `/v3/projects/eab92eed124d4c0782dfed9e27d104a7/users/23a561428bd946b681c52a8ce3799845/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.845374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69b6a3-9835-4c6f-9eff-89ae273c9110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab92eed124d4c0782dfed9e27d104a7, user_id=23a561428bd946b681c52a8ce3799845, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.848521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69b6a3-9835-4c6f-9eff-89ae273c9110 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.848714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1417/2844] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:39 2022] POST /identity/v3/users => generated 282 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:39.853674 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.864746 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ababf-f8e4-44bf-8840-4db292232c50 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.870311 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.871047 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.878325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.878325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.878325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.878325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.888173 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1418/2845] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:39.892912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.905615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9ccde-6ac7-49ee-be8c-65d07f4457bf admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.911229 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.911971 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.913728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.913728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.913728 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.915107 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:39.928166 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1419/2846] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:15:39.932965 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:39.946188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809a365-4f27-438f-add7-e61296027641 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:39.952764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e651b3-1a76-4ec7-b959-24a098337a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:39.953472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e651b3-1a76-4ec7-b959-24a098337a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:39.953764 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e651b3-1a76-4ec7-b959-24a098337a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:39.954092 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e651b3-1a76-4ec7-b959-24a098337a6f admin admin] PATH_INFO: `/v3/projects/df768956daa342c2a7cb7a1c0ae66b28/users/9b2641ea72fa48ed90dc1690ce8dd4f9/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:39.973607 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e651b3-1a76-4ec7-b959-24a098337a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df768956daa342c2a7cb7a1c0ae66b28, user_id=9b2641ea72fa48ed90dc1690ce8dd4f9, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:39.977039 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e651b3-1a76-4ec7-b959-24a098337a6f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:40.062909 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1428/2847] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:39 2022] PUT /identity/v3/projects/eab92eed124d4c0782dfed9e27d104a7/users/23a561428bd946b681c52a8ce3799845/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:40.075562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.075881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.076340 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.081982 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:40.089780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1420/2848] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:39 2022] PUT /identity/v3/projects/df768956daa342c2a7cb7a1c0ae66b28/users/9b2641ea72fa48ed90dc1690ce8dd4f9/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:40.100576 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.100937 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.101114 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.120210 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] MFA Rules not processed for user `23a561428bd946b681c52a8ce3799845`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:40.121934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:40.146223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] MFA Rules not processed for user `9b2641ea72fa48ed90dc1690ce8dd4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:40.147992 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:40.230495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1421/2849] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:40.236088 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1429/2850] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:40.242725 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:40.243009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:40.245062 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:40.346118 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a144ba7-abb1-4425-a5f7-14b393e44bbb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:40.362091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ce56a-12d2-4241-a570-0190d656287e None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:40.364553 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:40.366209 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.366739 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.367411 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.369124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:40.370091 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.370597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.371054 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.371951 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:40.374496 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:40.397460 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:40.401512 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:40.420188 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1430/2851] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:40.549434 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1422/2852] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:15:40 2022] POST /identity/v3/projects => generated 385 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:15:40.551326 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:40.572846 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab3fef-366b-4203-acac-bd14a4a4ad55 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:40.604571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:40.692521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:40.693564 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.694104 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.694591 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.695719 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:40.699249 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:40.836483 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1431/2853] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:15:40 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:15:40.842077 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:40.855204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0886404-f8e5-48e7-96f9-0f4daddb9c61 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:40.858525 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:40.859155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.859441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.860233 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.860405 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:40.862273 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:40.878422 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1423/2854] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:40.885090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:40.903651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cad58-ed4a-40ef-a2aa-b329854719b3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:40.918673 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:40.919970 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:40.920472 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:40.920978 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] PATH_INFO: `/v3/projects/5c43fa1b4a814c2a996b798544c3d25d/users/98ba5449eb874d808f5fcad1d1c15228/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:40.948484 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c43fa1b4a814c2a996b798544c3d25d, user_id=98ba5449eb874d808f5fcad1d1c15228, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:40.952277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:41.083747 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1432/2855] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:40 2022] PUT /identity/v3/projects/5c43fa1b4a814c2a996b798544c3d25d/users/98ba5449eb874d808f5fcad1d1c15228/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:41.088701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:41.100463 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e4971-7ed2-47fa-a0ef-80d3ce1d38b5 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:41.197188 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef5ba6-e4b0-4387-979d-f567857a8b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:41.198412 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef5ba6-e4b0-4387-979d-f567857a8b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:41.198820 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef5ba6-e4b0-4387-979d-f567857a8b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:41.199959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef5ba6-e4b0-4387-979d-f567857a8b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:41.201333 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef5ba6-e4b0-4387-979d-f567857a8b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:41.203226 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:41.205634 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef5ba6-e4b0-4387-979d-f567857a8b4b admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:41.218113 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1424/2856] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:15:41.233223 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:41.244857 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebba14-6b1a-4af4-ac17-2fce3c097cc7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:41.250410 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:41.251598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:41.251890 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:41.252174 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] PATH_INFO: `/v3/projects/5c43fa1b4a814c2a996b798544c3d25d/users/98ba5449eb874d808f5fcad1d1c15228/roles/718611f559c1495c92b36acf3fc8e1fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:41.254354 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c43fa1b4a814c2a996b798544c3d25d, user_id=98ba5449eb874d808f5fcad1d1c15228, role_id=718611f559c1495c92b36acf3fc8e1fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:41.257266 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:41.270724 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1433/2857] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:15:41 2022] PUT /identity/v3/projects/5c43fa1b4a814c2a996b798544c3d25d/users/98ba5449eb874d808f5fcad1d1c15228/roles/718611f559c1495c92b36acf3fc8e1fe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:15:41.283767 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:41.284165 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:41.284522 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:41.320830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] MFA Rules not processed for user `98ba5449eb874d808f5fcad1d1c15228`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:41.322704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:41.407373 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1425/2858] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:41.419848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:41.421746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:41.511396 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7f532-d077-47f3-afba-4ffb7c005caa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:41.517378 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:41.518100 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:41.518434 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:41.518901 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:41.599989 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:41.601795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:41.615359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1434/2859] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:41.718413 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:41.757055 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:41.759592 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:41.841065 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104389f-f5a7-4e01-a5d9-eb4bf175c0df None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:41.845098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:41.848132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:41.848132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:41.848132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:41.848644 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:41.953231 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:41.955214 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:41.968973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1426/2860] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:42.792303 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:43.538275 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:44.075317 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:44.727364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:48.225387 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:48.237153 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440e44-0552-40be-a24d-ae1fcd595dde service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:48.243862 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197ad7-61bd-43d4-9b80-be5f9c4ba303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:48.244946 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197ad7-61bd-43d4-9b80-be5f9c4ba303 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:48.245329 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197ad7-61bd-43d4-9b80-be5f9c4ba303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:48.245699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197ad7-61bd-43d4-9b80-be5f9c4ba303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:48.264675 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b197ad7-61bd-43d4-9b80-be5f9c4ba303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:48.267369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b197ad7-61bd-43d4-9b80-be5f9c4ba303 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:48.285802 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1435/2861] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:55.269270 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=29 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:57.482439 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:57.568944 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396b796-0413-4a9a-bc38-071d224b80d3 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:57.570355 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd299-f76d-488f-89ca-c9fc20235777 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:57.570911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd299-f76d-488f-89ca-c9fc20235777 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:57.571679 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd299-f76d-488f-89ca-c9fc20235777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:57.576814 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:57.577530 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:57.580361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:57.580361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] PATH_INFO: `/v3/users/23a561428bd946b681c52a8ce3799845` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:57.580361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a561428bd946b681c52a8ce3799845)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:57.581671 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:57.607713 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ed5bd299-f76d-488f-89ca-c9fc20235777 None None] MFA Rules not processed for user `176ac62de5974fef91b59293df718ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:57.609819 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ed5bd299-f76d-488f-89ca-c9fc20235777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:57.619227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:57.701288 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1436/2862] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:15:57.894033 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a561428bd946b681c52a8ce3799845'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:57.995419 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a561428bd946b681c52a8ce3799845'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:58.013067 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a561428bd946b681c52a8ce3799845'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:58.018841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a561428bd946b681c52a8ce3799845'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:58.023337 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a561428bd946b681c52a8ce3799845'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:58.032963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1427/2863] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:57 2022] DELETE /identity/v3/users/23a561428bd946b681c52a8ce3799845 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:58.039314 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:58.039840 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:58.040265 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:58.070232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:15:58.072728 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:58.151651 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1437/2864] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:15:58.165828 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:58.167679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:58.266477 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9475-708f-4f8c-b7b3-61aa01b748eb admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:58.272306 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:58.273098 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:58.273263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:58.275081 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:58.286041 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:58.287520 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:58.299784 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1428/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:58.932941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:58.934876 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:58.951506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d356fe-5f9b-487e-993b-f6042bfc4abc None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:58.958454 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:58.959549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:58.959977 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:58.960441 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] PATH_INFO: `/v3/projects/eab92eed124d4c0782dfed9e27d104a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:58.963131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab92eed124d4c0782dfed9e27d104a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:58.966011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:59.118561 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab92eed124d4c0782dfed9e27d104a7', 'request_id': 'req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.136708 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab92eed124d4c0782dfed9e27d104a7', 'request_id': 'req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.146836 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab92eed124d4c0782dfed9e27d104a7', 'request_id': 'req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.270200 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] The token cache is being invalidate because project eab92eed124d4c0782dfed9e27d104a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:15:59.270799 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.271957 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.274982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1438/2866] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:15:58 2022] DELETE /identity/v3/projects/eab92eed124d4c0782dfed9e27d104a7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:15:59.283819 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:59.305353 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154f5b8-0537-48ef-8240-ffcebe608a3c service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:59.344290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:59.421538 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:59.422826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:59.423342 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:59.423841 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] PATH_INFO: `/v3/users/98ba5449eb874d808f5fcad1d1c15228` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:59.425958 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ba5449eb874d808f5fcad1d1c15228)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:59.429887 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:59.761516 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ba5449eb874d808f5fcad1d1c15228'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.901606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ba5449eb874d808f5fcad1d1c15228'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.917830 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ba5449eb874d808f5fcad1d1c15228'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.918775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ba5449eb874d808f5fcad1d1c15228'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.927389 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-360829c2-040d-4e6c-96a7-3e225af1620e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ba5449eb874d808f5fcad1d1c15228'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:15:59.935075 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1429/2867] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:15:59 2022] DELETE /identity/v3/users/98ba5449eb874d808f5fcad1d1c15228 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:00.611795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:00.611795 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:00.626965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b88b-42f5-4cf9-b3e7-47ca32fefde2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:00.688391 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:00.719942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:00.722479 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:00.723090 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:00.723601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] PATH_INFO: `/v3/projects/5c43fa1b4a814c2a996b798544c3d25d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:00.725875 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c43fa1b4a814c2a996b798544c3d25d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:00.733477 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:00.901510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c43fa1b4a814c2a996b798544c3d25d', 'request_id': 'req-2d1de0f6-8360-4178-847b-f41b294dffd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:00.913012 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c43fa1b4a814c2a996b798544c3d25d', 'request_id': 'req-2d1de0f6-8360-4178-847b-f41b294dffd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:00.922489 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c43fa1b4a814c2a996b798544c3d25d', 'request_id': 'req-2d1de0f6-8360-4178-847b-f41b294dffd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:01.024538 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] The token cache is being invalidate because project 5c43fa1b4a814c2a996b798544c3d25d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:01.024982 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:01.025694 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2d1de0f6-8360-4178-847b-f41b294dffd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:01.027988 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1439/2868] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:00 2022] DELETE /identity/v3/projects/5c43fa1b4a814c2a996b798544c3d25d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:01.034352 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.035073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.035073 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.056996 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:01.058211 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:01.146507 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1430/2869] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:16:01.160691 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.166364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.166364 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.184233 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:01.185491 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:01.194093 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1440/2870] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:01.199376 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:01.211200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a9b77-0c85-4255-af41-f9163d6fa353 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:01.219252 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:01.220190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.220846 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.221247 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.222562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:16:01.223178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:01.227855 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:01.250861 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1431/2871] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:16:01.255566 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:01.268826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66aeb4-750d-4ec9-9e91-cdb7337ef5a7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:01.275706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:01.276584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.277037 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.277468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.278374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:01.280971 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:01.457161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1441/2872] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:16:01 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:16:01.460094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:01.472895 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b48738-aeae-4684-a3ff-c4603f9f903d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:01.505408 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:01.542269 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:01.543003 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.543431 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.543779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.544513 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:01.546640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:01.725832 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1432/2873] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:16:01 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:16:01.733173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:01.756518 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248056a-443e-433b-b883-20f71bb810fa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:01.766698 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:01.767778 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.768485 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.768878 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.769575 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:01.771433 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:01.794506 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1442/2874] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:16:01.796535 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:01.808466 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fd607-772a-4b13-b0e7-70f492479a91 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:01.813657 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d064f-2251-4879-944b-6bb1cbb66a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:01.814335 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d064f-2251-4879-944b-6bb1cbb66a71 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.815130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d064f-2251-4879-944b-6bb1cbb66a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.815130 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d064f-2251-4879-944b-6bb1cbb66a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.815531 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d064f-2251-4879-944b-6bb1cbb66a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:01.817163 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d064f-2251-4879-944b-6bb1cbb66a71 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:01.835175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1433/2875] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:16:01.837228 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:01.864336 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88abdf-b465-4d98-84a6-872f33f0db17 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:01.874069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:01.875247 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:01.875783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:01.876227 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] PATH_INFO: `/v3/projects/afd71f793b6146399e0c99d947098002/users/d0eb0f0c083046db991cad1b582063c8/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:01.896105 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd71f793b6146399e0c99d947098002, user_id=d0eb0f0c083046db991cad1b582063c8, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:01.907941 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:02.023420 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1443/2876] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:16:01 2022] PUT /identity/v3/projects/afd71f793b6146399e0c99d947098002/users/d0eb0f0c083046db991cad1b582063c8/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:02.051185 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:02.053581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:02.054446 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:02.115244 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] MFA Rules not processed for user `d0eb0f0c083046db991cad1b582063c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:02.116743 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:02.209053 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1434/2877] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:02.222072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:02.224500 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:02.308871 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1d71-33df-402d-b061-256c00767001 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:02.329217 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1bcf8-28b3-4d30-98a2-cc288f472eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:02.330746 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1bcf8-28b3-4d30-98a2-cc288f472eae service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:02.331230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1bcf8-28b3-4d30-98a2-cc288f472eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:02.331866 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1bcf8-28b3-4d30-98a2-cc288f472eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:02.360684 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1bcf8-28b3-4d30-98a2-cc288f472eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:02.363049 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1bcf8-28b3-4d30-98a2-cc288f472eae service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:02.382000 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1444/2878] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:16:02.505777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:02.604263 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354094b-a803-4626-912c-ec6c038891a8 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:02.609590 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:02.610395 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:02.610786 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:02.611122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] PATH_INFO: `/v3/users/d0eb0f0c083046db991cad1b582063c8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:02.612420 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0eb0f0c083046db991cad1b582063c8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:02.614612 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:02.976611 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:02.988329 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0eb0f0c083046db991cad1b582063c8'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:03.112124 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0eb0f0c083046db991cad1b582063c8'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:03.131398 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0eb0f0c083046db991cad1b582063c8'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:03.132223 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0eb0f0c083046db991cad1b582063c8'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:03.141851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0eb0f0c083046db991cad1b582063c8'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:03.168318 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1435/2879] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:16:02 2022] DELETE /identity/v3/users/d0eb0f0c083046db991cad1b582063c8 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:03.171700 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:03.172324 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:03.172721 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:03.207593 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:03.210218 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:03.217868 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:03.317257 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1445/2880] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:03.334917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:03.336629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:03.417345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93da5-5bfd-42fc-aadb-4f9eb7b4c7e1 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:03.423588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff9076-3e4e-40ec-9978-f11ed262e17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:03.423588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff9076-3e4e-40ec-9978-f11ed262e17b service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:03.423987 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff9076-3e4e-40ec-9978-f11ed262e17b service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:03.424162 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff9076-3e4e-40ec-9978-f11ed262e17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:03.439498 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff9076-3e4e-40ec-9978-f11ed262e17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:03.441176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff9076-3e4e-40ec-9978-f11ed262e17b service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:03.455442 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1436/2881] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:16:04.127588 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:04.129289 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:04.153469 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549885-cd1c-405e-88ec-77d8bf8a18be None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:04.161997 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:04.162657 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:04.162965 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:04.163558 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] PATH_INFO: `/v3/projects/afd71f793b6146399e0c99d947098002` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:04.165030 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd71f793b6146399e0c99d947098002)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:04.167302 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:04.345682 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd71f793b6146399e0c99d947098002', 'request_id': 'req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:04.355888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd71f793b6146399e0c99d947098002', 'request_id': 'req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:04.373975 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd71f793b6146399e0c99d947098002', 'request_id': 'req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:04.508944 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] The token cache is being invalidate because project afd71f793b6146399e0c99d947098002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:04.509317 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:04.510026 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:04.511552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1446/2882] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:04 2022] DELETE /identity/v3/projects/afd71f793b6146399e0c99d947098002 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:11.311374 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:11.312132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:11.312132 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:11.333871 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] MFA Rules not processed for user `93f843d15d194f0b9f44ec33c49b9b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:11.335154 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:11.406019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1437/2883] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:14.505013 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:18.660036 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:18.666617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:18.793240 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e49083-1e6e-4cc6-a4a7-0157b01f9cab admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:18.799732 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:18.800773 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:18.801576 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:18.802290 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:18.805976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:18.840113 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:18.841652 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa22ba-7e1e-49a0-b8e5-b072ade46780 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:18.856057 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1447/2884] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:16:21.397775 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:DELETE=5,keystone:SELECT=65,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:25.714549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:25.716744 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:25.807885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302e9f-0769-4ecd-b1c7-c1b81824e58c None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:25.815207 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:25.816290 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:25.816683 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:25.816967 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] PATH_INFO: `/v3/users/9b2641ea72fa48ed90dc1690ce8dd4f9` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:25.818935 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2641ea72fa48ed90dc1690ce8dd4f9)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:25.823144 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:25.937716 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54935a6-84e8-4829-b213-05b5bdb27035 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:25.938617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54935a6-84e8-4829-b213-05b5bdb27035 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:25.938617 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54935a6-84e8-4829-b213-05b5bdb27035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:26.000735 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d54935a6-84e8-4829-b213-05b5bdb27035 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:26.002147 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d54935a6-84e8-4829-b213-05b5bdb27035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:26.082248 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1448/2885] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:16:26.104375 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2641ea72fa48ed90dc1690ce8dd4f9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:26.219638 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2641ea72fa48ed90dc1690ce8dd4f9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:26.245759 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2641ea72fa48ed90dc1690ce8dd4f9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:26.246923 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2641ea72fa48ed90dc1690ce8dd4f9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:26.257268 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2641ea72fa48ed90dc1690ce8dd4f9'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:26.267176 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1438/2886] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:16:25 2022] DELETE /identity/v3/users/9b2641ea72fa48ed90dc1690ce8dd4f9 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:26.272789 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:26.274261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:26.274761 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:26.291035 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:26.293963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:26.305366 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:26.308769 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:26.403888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1449/2887] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:26.413114 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:26.415841 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:26.439384 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:26.487804 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:26.496175 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f4c6-c2ba-473b-96bb-d44bc21da86f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:26.504624 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:26.505383 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:26.506045 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:26.506436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:26.510526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c15e7-a24c-4ca7-b067-dcc6da994f54 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:26.510773 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:26.512458 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:26.513374 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:26.513959 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:26.514439 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:26.527159 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:26.529031 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:26.543693 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1450/2888] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:16:26.616556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:26.619482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049da5-bdc1-4c8f-9e39-397d0a02eedb service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:26.633447 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1439/2889] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:16:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:16:27.223237 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:27.225924 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:27.282925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6fee-f7fc-4fc2-b32b-5967432a4de7 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:27.295942 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:27.297598 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:27.299643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:27.301309 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] PATH_INFO: `/v3/projects/df768956daa342c2a7cb7a1c0ae66b28` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:27.308440 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df768956daa342c2a7cb7a1c0ae66b28)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:27.314478 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:27.523771 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df768956daa342c2a7cb7a1c0ae66b28', 'request_id': 'req-3e4eae92-b277-48c4-9a06-e58b646fc53a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:27.532992 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df768956daa342c2a7cb7a1c0ae66b28', 'request_id': 'req-3e4eae92-b277-48c4-9a06-e58b646fc53a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:27.544549 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df768956daa342c2a7cb7a1c0ae66b28', 'request_id': 'req-3e4eae92-b277-48c4-9a06-e58b646fc53a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:27.661166 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] The token cache is being invalidate because project df768956daa342c2a7cb7a1c0ae66b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:27.665533 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:27.667277 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3e4eae92-b277-48c4-9a06-e58b646fc53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:27.672925 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1451/2890] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:27 2022] DELETE /identity/v3/projects/df768956daa342c2a7cb7a1c0ae66b28 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:27.679137 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:27.680493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:27.680493 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:27.703190 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:27.704826 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:27.805146 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1440/2891] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:27.814685 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:27.815161 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:27.815628 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:27.849430 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:27.852119 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:27.865126 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1452/2892] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:27.871571 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:27.889396 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75322c7e-86cd-428d-9d42-48c3a926375d None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:27.897305 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:27.900090 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:27.900505 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:27.900900 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:27.901733 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81705) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:16:27.902202 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:27.904837 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:27.928715 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1441/2893] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:16:27.936526 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:27.947827 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec1b35-461a-4e97-8683-2501aafd1ccd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:27.955730 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:27.956979 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:27.957284 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:27.958262 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:27.959521 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:27.963389 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:28.118910 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1453/2894] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:16:27 2022] POST /identity/v3/projects => generated 371 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:16:28.125105 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:28.144609 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9ebc8-be4f-42db-a688-d7abd7884e09 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:28.261938 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:28.263020 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:28.263518 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:28.264125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] PATH_INFO: `/v3/users` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:28.265147 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:28.268598 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:28.399178 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1442/2895] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:16:28 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:16:28.405834 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:28.426376 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c877b-81d4-4b62-8c16-de75a8fcdc3a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:28.437235 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:28.438777 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:28.439560 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:28.440321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:28.441832 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:28.445972 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:28.468794 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1454/2896] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:16:28.473822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:28.490822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2b6e-cc0a-4cf0-ac4f-9927d4563041 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:28.504106 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc399d4-bf0f-4e95-b519-5b3bd82f64c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:28.504956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc399d4-bf0f-4e95-b519-5b3bd82f64c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:28.505393 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc399d4-bf0f-4e95-b519-5b3bd82f64c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:28.506069 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc399d4-bf0f-4e95-b519-5b3bd82f64c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:28.507704 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc399d4-bf0f-4e95-b519-5b3bd82f64c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:28.508889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc399d4-bf0f-4e95-b519-5b3bd82f64c5 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:28.526155 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1443/2897] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:16:28.532046 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:28.557025 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a2b6-1e12-46e5-b0ae-0874cbd3a5a8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:28.565359 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:28.566094 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:28.566388 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:28.566672 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] PATH_INFO: `/v3/projects/4c85d259196745588a4569fb7766d758/users/aed90e7ff3924f5495935b7c9b6c89cd/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:28.606382 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c85d259196745588a4569fb7766d758, user_id=aed90e7ff3924f5495935b7c9b6c89cd, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:28.611625 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:28.762916 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1455/2898] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:16:28 2022] PUT /identity/v3/projects/4c85d259196745588a4569fb7766d758/users/aed90e7ff3924f5495935b7c9b6c89cd/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:16:28.764024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:28.764792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:28.765503 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:28.811770 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:28.832220 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] MFA Rules not processed for user `aed90e7ff3924f5495935b7c9b6c89cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:28.833776 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:28.962620 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:28.964645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:28.966024 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1444/2899] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:16:29.059904 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:29.096076 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d709b07-91e1-4a88-8cb0-397c9d521101 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:29.106097 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb8c3c-af61-4b1b-8560-22e058cad5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:29.107466 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb8c3c-af61-4b1b-8560-22e058cad5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:29.107902 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb8c3c-af61-4b1b-8560-22e058cad5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:29.108343 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb8c3c-af61-4b1b-8560-22e058cad5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:29.130649 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb8c3c-af61-4b1b-8560-22e058cad5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:29.134461 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb8c3c-af61-4b1b-8560-22e058cad5f0 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:29.156513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1456/2900] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:16:29.231489 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:29.424164 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cebdc-a097-47cc-8ef5-84071303d136 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:29.431204 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:29.432241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:29.432727 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:29.433228 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] PATH_INFO: `/v3/users/aed90e7ff3924f5495935b7c9b6c89cd` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:29.434847 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed90e7ff3924f5495935b7c9b6c89cd)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:29.437616 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:30.015724 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed90e7ff3924f5495935b7c9b6c89cd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:30.231615 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed90e7ff3924f5495935b7c9b6c89cd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:30.257726 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed90e7ff3924f5495935b7c9b6c89cd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:30.260097 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed90e7ff3924f5495935b7c9b6c89cd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:30.299969 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed90e7ff3924f5495935b7c9b6c89cd'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:30.330553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1445/2901] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:16:29 2022] DELETE /identity/v3/users/aed90e7ff3924f5495935b7c9b6c89cd => generated 0 bytes in 1100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:30.337596 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:30.338315 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:30.338584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:30.371751 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:30.374643 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:30.513670 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1457/2902] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:30.535472 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:30.537088 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:30.595964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:30.696139 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05984d5-57ab-4f0a-96ec-dffc66cf060d admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:30.702359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:30.703322 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:30.703851 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:30.704706 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:30.719680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:30.721549 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:30.738486 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1446/2903] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:16:32.168131 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:32.170407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:32.200120 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd3ba4-3be7-4a03-bfa9-ac0b412065df None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:32.207334 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:32.208175 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:32.208464 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:32.208995 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] PATH_INFO: `/v3/projects/4c85d259196745588a4569fb7766d758` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:32.210884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c85d259196745588a4569fb7766d758)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:32.213009 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:32.443087 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c85d259196745588a4569fb7766d758', 'request_id': 'req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:32.457127 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c85d259196745588a4569fb7766d758', 'request_id': 'req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:32.468208 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c85d259196745588a4569fb7766d758', 'request_id': 'req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:32.613571 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] The token cache is being invalidate because project 4c85d259196745588a4569fb7766d758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:32.614143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:32.614927 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-76b9ebbe-fb43-4d4c-b89b-69bb3ef68fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:32.616718 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1458/2904] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:32 2022] DELETE /identity/v3/projects/4c85d259196745588a4569fb7766d758 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:38.733588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:38.735390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:38.865031 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae88e-0350-4bf3-b73c-ac341b488eeb service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:39.000345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:39.002019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:39.002504 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:39.003018 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:39.008345 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:39.026714 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:39.028782 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:39.048997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1447/2905] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:16:42.608070 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:47.042395 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:47.043260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:47.043260 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:47.066503 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] MFA Rules not processed for user `a910227bf388410196f8b201db25ca1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:47.068653 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:47.163604 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1459/2906] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 14:16:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:16:47.177025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:47.178959 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:47.233255 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=100 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:47.260677 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d60e1d-1204-4f2e-bd87-38b9af4cd03b service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:47.266441 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0506db33e0e47aaa47a9a3fd9bc2f98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:47.267125 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:47.267679 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:47.267963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:47.279063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:47.280700 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:47.291194 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1448/2907] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 14:16:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:16:51.152621 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:51.154967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:51.265404 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72126f1-0193-415b-9719-4c707dde3470 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:51.270919 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:51.271171 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:51.271510 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:51.271888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] PATH_INFO: `/v3/users/176ac62de5974fef91b59293df718ebc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:51.273816 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=176ac62de5974fef91b59293df718ebc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:51.275812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:51.524135 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176ac62de5974fef91b59293df718ebc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:51.524886 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176ac62de5974fef91b59293df718ebc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:51.540071 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176ac62de5974fef91b59293df718ebc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:51.645321 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176ac62de5974fef91b59293df718ebc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:51.655563 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-01bd89d0-eaba-46d7-9b64-fa3ed31dd34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176ac62de5974fef91b59293df718ebc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:51.662562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1460/2908] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:16:51 2022] DELETE /identity/v3/users/176ac62de5974fef91b59293df718ebc => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:51.979200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:51.997133 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6dcf7-36cb-4e69-ab90-6cd1655a6a1b service swift] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:52.094997 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:52.095629 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:52.095956 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:52.096275 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] PATH_INFO: `/v3/projects/493732d3c282440f8593558858bb0b33` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:52.097760 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] RBAC: Authorizing `identity:delete_project(project_id=493732d3c282440f8593558858bb0b33)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:52.099912 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:52.252444 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493732d3c282440f8593558858bb0b33', 'request_id': 'req-09f86d5f-b709-4567-9670-cbca7d07abca'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:52.260930 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493732d3c282440f8593558858bb0b33', 'request_id': 'req-09f86d5f-b709-4567-9670-cbca7d07abca'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:52.385361 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493732d3c282440f8593558858bb0b33', 'request_id': 'req-09f86d5f-b709-4567-9670-cbca7d07abca'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:52.399497 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] The token cache is being invalidate because project 493732d3c282440f8593558858bb0b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:52.399842 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:52.400658 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-09f86d5f-b709-4567-9670-cbca7d07abca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:52.402390 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1449/2909] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:51 2022] DELETE /identity/v3/projects/493732d3c282440f8593558858bb0b33 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:52.407845 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:52.408601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:52.408601 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:52.430835 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:52.432256 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:52.455587 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:52.504654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1461/2910] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:52.513821 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:52.514170 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:52.514457 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:52.537668 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:52.539126 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:52.547633 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1450/2911] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:16:52.552370 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:52.566999 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37de22-99db-488b-a883-bcfc67d6937a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:52.575688 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:52.576495 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:52.576888 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:52.577274 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:52.578316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:16:52.579121 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:52.580967 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:52.595788 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1462/2912] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 14:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:16:52.601063 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:52.612356 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8180709-fade-4f75-996f-d12dc93b9fd3 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:52.617866 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:52.618604 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:52.618889 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:52.619167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:52.619822 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:52.621779 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:52.777351 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1451/2913] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 14:16:52 2022] POST /identity/v3/projects => generated 397 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:16:52.783937 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:52.794922 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a25717-63f1-4b47-acec-e0b224f1b8e6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:52.860826 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:52.861513 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:52.861860 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:52.862306 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:52.863083 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:52.865365 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:53.083677 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1463/2914] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 14:16:52 2022] POST /identity/v3/users => generated 299 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:16:53.087709 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:53.098780 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958dbb4-7a73-4d03-8554-db5ff4007782 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:53.106197 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:53.107093 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:53.107553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:53.107999 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:53.108954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:53.111556 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:53.126436 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1452/2915] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:16:53.131177 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:53.142998 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b136e2d-a68b-4bcc-8f63-c8a1bcea2baf admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:53.148345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167dad-1a78-4fea-9c1f-52066364d7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:53.148990 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167dad-1a78-4fea-9c1f-52066364d7df admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:53.149272 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167dad-1a78-4fea-9c1f-52066364d7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:53.149552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167dad-1a78-4fea-9c1f-52066364d7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:53.150261 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167dad-1a78-4fea-9c1f-52066364d7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:53.151889 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167dad-1a78-4fea-9c1f-52066364d7df admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:53.173481 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1464/2916] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 14:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:16:53.177831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:53.190873 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd87b5-0782-460b-8c26-a1457c309b89 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:53.196562 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:53.197248 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] REQUEST_METHOD: `PUT` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:53.197534 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:53.197815 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] PATH_INFO: `/v3/projects/ce4f5642f90247809fb105c3d413d5e8/users/ee22f236313d454b9a7d126fd395243d/roles/e2d854cafb8b4baab26d81a4c699e2b4` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:53.216684 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4f5642f90247809fb105c3d413d5e8, user_id=ee22f236313d454b9a7d126fd395243d, role_id=e2d854cafb8b4baab26d81a4c699e2b4)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:53.219588 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:53.340514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:53.346836 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1453/2917] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 14:16:53 2022] PUT /identity/v3/projects/ce4f5642f90247809fb105c3d413d5e8/users/ee22f236313d454b9a7d126fd395243d/roles/e2d854cafb8b4baab26d81a4c699e2b4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:16:53.359039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:53.359456 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:53.359870 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:53.393072 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] MFA Rules not processed for user `ee22f236313d454b9a7d126fd395243d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:16:53.394486 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:53.470532 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1465/2918] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:16:53 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:16:53.487150 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:53.488952 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:53.555614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c2c71-3b71-4429-91b3-213cf8f2717f admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:53.559519 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdd369-4a0d-4965-a0be-8f345bf44e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:53.560210 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd369-4a0d-4965-a0be-8f345bf44e43 service cinder] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:53.560777 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd369-4a0d-4965-a0be-8f345bf44e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:53.561179 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd369-4a0d-4965-a0be-8f345bf44e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:53.572160 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdd369-4a0d-4965-a0be-8f345bf44e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:53.574100 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdd369-4a0d-4965-a0be-8f345bf44e43 service cinder] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:53.586581 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1454/2919] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:53 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:17:03.211555 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:17:03.236310 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:17:03.285709 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd15055-57f0-476e-96e3-fa17446bd2cd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:17:03.291468 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:17:03.292181 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:17:03.292505 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:17:03.292818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] PATH_INFO: `/v3/users/ee22f236313d454b9a7d126fd395243d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:17:03.294232 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee22f236313d454b9a7d126fd395243d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:17:03.296361 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:17:03.580511 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=27 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:17:03.611729 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee22f236313d454b9a7d126fd395243d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:03.612792 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee22f236313d454b9a7d126fd395243d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:03.634976 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee22f236313d454b9a7d126fd395243d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:03.748241 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee22f236313d454b9a7d126fd395243d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:03.757567 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee22f236313d454b9a7d126fd395243d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:03.764353 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1466/2920] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:17:03 2022] DELETE /identity/v3/users/ee22f236313d454b9a7d126fd395243d => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:17:03.769241 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:17:03.769536 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:17:03.769792 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:17:03.791926 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] MFA Rules not processed for user `c6e62ba856084972b01892225cf77493`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:17:03.793169 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:17:03.891680 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1455/2921] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:17:03.903011 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:17:03.904676 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:17:04.000196 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754be7c-4233-479e-96cc-4770547a8d0d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:17:04.007039 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:17:04.008133 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:17:04.008624 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:17:04.009108 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:17:04.026294 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:17:04.029138 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:17:04.047881 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1467/2922] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:17:04.625961 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:17:04.629216 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:17:04.651885 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c323e3b-b67d-4441-b624-4db6c33fd5b0 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:17:04.659947 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:17:04.661094 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:17:04.661595 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:17:04.662096 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] PATH_INFO: `/v3/projects/ce4f5642f90247809fb105c3d413d5e8` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:17:04.664473 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4f5642f90247809fb105c3d413d5e8)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:17:04.666606 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:17:04.993037 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4f5642f90247809fb105c3d413d5e8', 'request_id': 'req-7789cb31-3473-4e55-b4e8-a4c5dbe99696'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:05.000282 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4f5642f90247809fb105c3d413d5e8', 'request_id': 'req-7789cb31-3473-4e55-b4e8-a4c5dbe99696'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:05.122145 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4f5642f90247809fb105c3d413d5e8', 'request_id': 'req-7789cb31-3473-4e55-b4e8-a4c5dbe99696'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:05.134701 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] The token cache is being invalidate because project ce4f5642f90247809fb105c3d413d5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:17:05.135048 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:05.135667 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-7789cb31-3473-4e55-b4e8-a4c5dbe99696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:17:05.137281 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1456/2923] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:17:04 2022] DELETE /identity/v3/projects/ce4f5642f90247809fb105c3d413d5e8 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:17:14.039316 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:17:15.131123 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:18:13.055358 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:18:13.057173 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:13.122562 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c1d3-91eb-4403-9970-d802dcbb849c service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:18:13.183435 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f9ea2ca628427bb165e1da9c0d1c7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:18:13.184958 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:13.185457 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:13.186068 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:13.189884 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:13.287300 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:18:13.289106 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:18:13.300271 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1468/2924] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:18:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:18:22.663404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] REQUEST_METHOD: `POST` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:22.663404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:22.663404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:22.663404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.auth.core [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] MFA Rules not processed for user `93f843d15d194f0b9f44ec33c49b9b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81705) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:18:22.663404 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:22.667187 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1457/2925] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 14:18:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:18:23.069635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:18:23.071150 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:23.156765 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677947d-84ae-42cd-b0dd-0ffa748f14c7 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:18:23.161537 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:18:23.162198 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:23.162487 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:23.162767 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:23.164645 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:23.176345 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:18:23.177818 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:18:23.187522 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1469/2926] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:18:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:18:24.281964 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:18:24.285748 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:24.373019 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcdd6a-dbf6-4925-b2ad-686a06b44a18 None None] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:18:24.390167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:18:24.390167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:24.390167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:24.390167 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:24.391042 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:24.397675 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:18:24.400988 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:18:24.418005 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1458/2927] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:18:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:18:33.183230 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:18:34.412391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=35 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:18:39.271285 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:18:39.274014 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:39.407823 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c04367-4ea0-4bec-a094-959af9696ae0 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:18:39.418637 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:18:39.419689 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:39.420220 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:39.420725 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:39.423994 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:39.446480 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:18:39.450641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:18:39.470341 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1470/2928] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:18:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:18:45.537963 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:18:45.539541 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:45.557540 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc33aae-8708-4cb8-a506-d3c29e1142c4 service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:18:45.566986 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf5c9c7caaf4451a9b1ca1a1095408d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:18:45.567640 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:45.567917 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:45.568200 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:45.570289 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:45.586313 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:18:45.588381 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:18:45.608954 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1459/2929] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:18:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:18:49.461577 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=34 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:18:51.445297 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:18:51.447016 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:51.465282 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb38ee-669d-4298-bb3b-53208b39fdc6 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:18:51.474213 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:18:51.475307 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:18:51.475828 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:18:51.476418 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:18:51.479475 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:18:51.501759 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:18:51.505552 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:18:51.525081 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1471/2930] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:18:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:18:55.598040 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=6 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:19:01.516812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:20:33.555553 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:33.557380 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:33.570811 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8525730-85ad-4586-9a7c-6418aef6d965 service nova] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:33.577015 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eeb9414ed49428684b9028c18476bba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:33.577977 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:33.578409 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:33.578831 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:33.581514 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:33.665495 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:33.666903 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:33.678974 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1460/2931] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:20:38.433779 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:38.451654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ffe9-a079-47a2-a4ed-d1b1708c39af service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:38.458164 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:38.458812 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:38.459092 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:38.459369 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:38.461633 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:38.475699 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:38.477541 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:38.491726 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1472/2932] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:20:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:20:43.249025 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:43.263614 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9c53-89f9-4899-a750-658a58adff73 service neutron] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:43.272297 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:43.273391 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] REQUEST_METHOD: `GET` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:43.273916 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:43.274480 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:43.277502 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:43.300423 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:43.303050 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:43.323482 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1461/2933] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:20:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:20:43.429696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:43.431770 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:43.448848 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade21c-149e-4a57-bedf-2515ec70ed2d service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:43.456091 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:43.457453 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:43.458069 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:43.458584 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] PATH_INFO: `/v3/users/93f843d15d194f0b9f44ec33c49b9b4c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:43.460201 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f843d15d194f0b9f44ec33c49b9b4c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:43.462934 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:43.694923 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f843d15d194f0b9f44ec33c49b9b4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:43.695654 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f843d15d194f0b9f44ec33c49b9b4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:43.711033 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f843d15d194f0b9f44ec33c49b9b4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:43.819497 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f843d15d194f0b9f44ec33c49b9b4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:43.828470 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f843d15d194f0b9f44ec33c49b9b4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:43.835960 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1473/2934] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:20:43 2022] DELETE /identity/v3/users/93f843d15d194f0b9f44ec33c49b9b4c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:20:44.141494 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:44.176122 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb81887-c2a1-408d-9874-16b958f3d2c9 service placement] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:44.287560 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:44.288459 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:44.288860 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:44.289260 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] PATH_INFO: `/v3/projects/8acdb976501045c0813d16f3d0ffd13d` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:44.290973 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8acdb976501045c0813d16f3d0ffd13d)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:44.293984 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:44.493161 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8acdb976501045c0813d16f3d0ffd13d', 'request_id': 'req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.500589 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8acdb976501045c0813d16f3d0ffd13d', 'request_id': 'req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.629544 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8acdb976501045c0813d16f3d0ffd13d', 'request_id': 'req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.642844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] The token cache is being invalidate because project 8acdb976501045c0813d16f3d0ffd13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:20:44.646844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.646844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.646844 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1462/2935] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:20:44 2022] DELETE /identity/v3/projects/8acdb976501045c0813d16f3d0ffd13d => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:20:44.650339 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:44.652080 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:44.667933 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd6db1-1312-479d-9c74-4a1b584dea36 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:44.738424 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:44.739075 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:44.739360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:44.739641 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] PATH_INFO: `/v3/users/f9af38df76824077b4133fa9b8150a5f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:44.740893 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9af38df76824077b4133fa9b8150a5f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:44.742911 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:44.956666 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9af38df76824077b4133fa9b8150a5f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.957431 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9af38df76824077b4133fa9b8150a5f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:44.971065 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:20:44.973847 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9af38df76824077b4133fa9b8150a5f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.042407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9af38df76824077b4133fa9b8150a5f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.053407 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9af38df76824077b4133fa9b8150a5f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.061855 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1474/2936] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 14:20:44 2022] DELETE /identity/v3/users/f9af38df76824077b4133fa9b8150a5f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:20:45.348771 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Authenticating user token {{(pid=81705) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:20:45.350461 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.fernet_utils [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81705) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:20:45.369070 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093c885-42dc-41a6-a4cb-4dbc7bdc8584 admin admin] Validating token access rules against request {{(pid=81705) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:20:45.378543 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:20:45.462286 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e62ba856084972b01892225cf77493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cbeb42d06b34679b1e34ad6fea416bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81705) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:20:45.463490 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:20:45.464325 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:20:45.465076 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] PATH_INFO: `/v3/projects/3a8e2734029f4bf49339ca13ef88ee29` {{(pid=81705) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:20:45.468856 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a8e2734029f4bf49339ca13ef88ee29)` {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:20:45.472548 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] RBAC: Authorization granted {{(pid=81705) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:20:45.581672 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a8e2734029f4bf49339ca13ef88ee29', 'request_id': 'req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.591737 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a8e2734029f4bf49339ca13ef88ee29', 'request_id': 'req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.676225 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a8e2734029f4bf49339ca13ef88ee29', 'request_id': 'req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920'} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.690359 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] The token cache is being invalidate because project 3a8e2734029f4bf49339ca13ef88ee29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81705) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:20:45.693597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.693597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG keystone.notifications [None req-c8b62ea6-917d-4fb9-b53c-cf2bbc961920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81705) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:20:45.693597 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: [pid: 81705|app: 0|req: 1463/2937] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 14:20:45 2022] DELETE /identity/v3/projects/3a8e2734029f4bf49339ca13ef88ee29 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:20:55.110805 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:20:55.686449 dv-cinder-852211-7-iscsi devstack@keystone.service[81705]: DEBUG dbcounter [-] [81705] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81705) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:23:57.383780 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:23:57.385444 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:23:57.603696 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e81c6-47e5-483d-a3f9-31e46a95d5be admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:23:57.676592 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51e7d81890cc4ee5a357765f2a2bbbe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522bb405d1c44313b55c3c750fa31613', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:23:57.677143 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:23:57.677429 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:23:57.677706 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:23:57.679683 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:23:57.694148 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:23:57.695783 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b75e8e-5113-426d-8d4e-d3d37522f6cc service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:23:57.708635 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1475/2938] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:24:07.702360 dv-cinder-852211-7-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}